{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Save and load checkpoints\n",
    "\n",
    "This guide demonstrates how to save and load Flax NNX model checkpoints with [Orbax](https://orbax.readthedocs.io/).\n",
    "\n",
    "> **Note:** The Flax team does not actively maintain a library for saving and loading model checkpoints to disk. Therefore, it is recommended you use external libraries like [Orbax](https://orbax.readthedocs.io/en/latest/index.html) to do it.\n",
    "\n",
    "In this guide you will learn how to:\n",
    "\n",
    "* Save checkpoints.\n",
    "* Restore checkpoints.\n",
    "* Restore checkpoints if checkpoint structures differ. \n",
    "* Perform multi-process checkpointing. \n",
    "\n",
    "The Orbax API examples used throughout the guide are for demonstration purposes, and for the most up-to-date recommended APIs refer to the [Orbax website](https://orbax.readthedocs.io/).\n",
    "\n",
    "> **Note:** The Flax team recommends using [Orbax](https://orbax.readthedocs.io/en/latest/index.html) for saving and loading checkpoints to disk, as we do not actively maintain a library for these functionalities.\n",
    "\n",
    "> **Note:** If you are looking for Flax Linen's legacy `flax.training.checkpoints` package, it was deprecated in 2023 in favor of Orbax. The documentation resides on the [Flax Linen site](https://flax-linen.readthedocs.io/en/latest/guides/training_techniques/use_checkpointing.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Setup\n",
    "\n",
    "Import the necessary dependencies, set up a checkpoint directory and an example Flax NNX model - `TwoLayerMLP` - by subclassing [`nnx.Module`](https://flax.readthedocs.io/en/latest/api_reference/flax.nnx/module.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from flax import nnx\n",
    "import orbax.checkpoint as ocp\n",
    "import jax\n",
    "from jax import numpy as jnp\n",
    "import numpy as np\n",
    "\n",
    "ckpt_dir = ocp.test_utils.erase_and_create_empty('/tmp/my-checkpoints/')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "class TwoLayerMLP(nnx.Module):\n",
    "  def __init__(self, dim, rngs: nnx.Rngs):\n",
    "    self.linear1 = nnx.Linear(dim, dim, rngs=rngs, use_bias=False)\n",
    "    self.linear2 = nnx.Linear(dim, dim, rngs=rngs, use_bias=False)\n",
    "\n",
    "  def __call__(self, x):\n",
    "    x = self.linear1(x)\n",
    "    return self.linear2(x)\n",
    "\n",
    "# Instantiate the model and show we can run it.\n",
    "model = TwoLayerMLP(4, rngs=nnx.Rngs(0))\n",
    "x = jax.random.normal(jax.random.key(42), (3, 4))\n",
    "assert model(x).shape == (3, 4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save checkpoints\n",
    "\n",
    "JAX checkpointing libraries, such as Orbax, can save and load any given JAX [pytree](https://jax.readthedocs.io/en/latest/pytrees.html), which is a pure, possibly nested container of [`jax.Array`s)](https://jax.readthedocs.io/en/latest/key-concepts.html#jax-arrays-jax-array) (or, \"tensors\" as some other frameworks would put it). In the context of machine learning, the checkpoint is usually a pytree of model parameters and other data, such as optimizer states.\n",
    "\n",
    "In Flax NNX, you can obtain such a pytree from an [`nnx.Module`](https://flax.readthedocs.io/en/latest/api_reference/flax.nnx/module.html) by calling [`nnx.split`](https://flax.readthedocs.io/en/latest/api_reference/flax.nnx/graph.html#flax.nnx.split), and picking up the returned [`nnx.State`](https://flax.readthedocs.io/en/latest/api_reference/flax.nnx/state.html#flax.nnx.State)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<script> (()=>{ if (customElements.get('treescope-container') === undefined) { class TreescopeContainer extends HTMLElement { constructor() { super(); this.attachShadow({mode: \"open\"}); this.defns = {}; this.state = {}; } } customElements.define(\"treescope-container\", TreescopeContainer); } if (customElements.get('treescope-run-here') === undefined) { class RunHere extends HTMLElement { constructor() { super() } connectedCallback() { const run = child => { const fn = new Function(child.textContent); child.textContent = \"\"; fn.call(this); this.remove(); }; const child = this.querySelector(\"script\"); if (child) { run(child); } else { new MutationObserver(()=>{ run(this.querySelector(\"script\")); }).observe(this, {childList: true}); } } } customElements.define(\"treescope-run-here\", RunHere); } })(); </script> <treescope-container class=\"treescope_out_4f82ceeefe1c4ecd9cc4f9e1f20d1deb\" style=\"display:block\"></treescope-container> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_4f82ceeefe1c4ecd9cc4f9e1f20d1deb\")) .filter((elt) => !elt.dataset.setup) )[0]; root.dataset.setup = 1; const msg = document.createElement(\"span\"); msg.style = \"color: #cccccc; font-family: monospace;\"; msg.textContent = \"(Loading...)\"; root.state.loadingMsg = msg; root.shadowRoot.appendChild(msg); root.state.chain = new Promise((resolve, reject) => { const observer = new IntersectionObserver((entries) => { for (const entry of entries) { if (entry.isIntersecting) { resolve(); observer.disconnect(); return; } } }, {rootMargin: \"1000px\"}); window.setTimeout(() => { observer.observe(root); }, 0); }); root.state.deferring = false; const _insertNode = (node) => { for (let oldScript of node.querySelectorAll(\"script\")) { let newScript = document.createElement(\"script\"); newScript.type = oldScript.type; newScript.textContent = oldScript.textContent; oldScript.parentNode.replaceChild(newScript, oldScript); } if (root.state.loadingMsg) { root.state.loadingMsg.remove(); root.state.loadingMsg = null; } root.shadowRoot.appendChild(node); }; root.defns.insertContent = ((contentNode, compressed) => { if (compressed) { root.state.deferring = true; } if (root.state.deferring) { root.state.chain = (async () => { await root.state.chain; if (compressed) { const encoded = contentNode.textContent; const blob = new Blob([ Uint8Array.from(atob(encoded), (m) => m.codePointAt(0)) ]); const reader = blob.stream().pipeThrough( new DecompressionStream(\"deflate\") ).pipeThrough( new TextDecoderStream(\"utf-8\") ).getReader(); const parts = []; while (true) { const step = await reader.read(); if (step.done) { break; } parts.push(step.value); } const tpl = document.createElement('template'); tpl.innerHTML = parts.join(\"\"); _insertNode(tpl.content); } else { _insertNode(contentNode.content); } })(); } else { _insertNode(contentNode.content); } }); </script></treescope-run-here><div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtWY1X2sgW/1em6XkLPCWG8CWonBeQL1u1iq3W3T3skEzCSJjEZABxj//7u5OE72ht67bd3eI5ApM79/veub9h3+dTm1Rk7hHi645Lup7jcPQnch2fcuqwMvKIjTkdkz1kOoynTTyk9rSMhg5zfBfrsD7pU07SwZcycj1YsanP0wHrNJ+6sMocBss9rA8szxkxI607tuOVw617KPrWs4EA+FGD98vIpBzIGCeM76EhZeloPaMo/wFezl3ap/eUWbDP8QzipWFpD7nYMGAxbROTl5Gq94U2jKT7hFp9WMnIeSGPcUzBuDn/6EN6TH3aozblYCIecWdOm6aMe5T5VBdiSfg0suthfyf04/7cj2lvxECmB2u+7lGXI+GIgwR2XZvqWLh2x9E5EW7yCB4mKslk6qACngd5PkcGMZmPDhDvU1+2CD+HsJw4Bkmm5L7jczl4DqYRjrouYcJkTRdcxaZff4970sLMsAk8ZiPb3gslyKBmx3EYrCYnjjdIoWUdnEtYEo9WljnVxaJLPNPxhpjpRGbOJJkKEgEEJDeeoHS4aR9l1RTwoSZKrmkt24RZvI8ODpAiSJ5U3SN85DHwOyK2TxaK9UdMaLbO2u9Tkwv9AgLx4QH+HpGQhPRjhjORPXI7Ij7XGB0G4Wp4eEiSoU9SgsfehiB35PdDN+7F2DgTcRCa8YSVz9dBaBEGkjuWZYfl2w1KDLLVFbzECrH5NiJjSPAokkK74Ls8IFPhdMmThEIRsazb2PffQhVHfJPSnGd3CGkozYQ/pMCfkP5Bjlf2d+IKwKBjFDA8kFb7jIQ47oGl5O5AUiQoXY9vkjgMVARnMHj0VDHEeyAp9sxsl6AYw34XNJwu7vU8Mg7yJ+g/rwu7KlYUsCoi0J3hEDYuUeDgJYxfI8Fl5vBkue+MiZeKoY/IfdFErGWG2Uw+kxcEYAXxPGJ0XWiDpO/Y0NOWCYviL+rCgRllRDmGbiI2m0COe2A+A7880r9B5xWybugxoDaoD0Knsz69TogqyMY9YpfLPQKVTZa00oPXXqy8sAWnM6IHR70bfDuXRVnQmHu2I5r+ozIDj25KNrA38Am2IFvY5u4gGGtLfewnKwHPSqwfwgDqfaIPiJFKof+mFjqIrfGbZvQrGgZnSRklflPzPT3xPdVb3fSokoVvoKSIoxA88nwRQNeBE5V4MXKp/3Jig1IIBKWDHuA/luMvI3VhHid3fFOKTP2uST2fdx3WFekfU1pPlZKs5kU1xYYKfbX6YcTXVRRWDbFnwQwUqhEU9MNXSoN+6E57I85h+IhrQIvHcUkrIWmNChwJE2g88W8kkzOktQE2cYwhKyi2UWc67Dm2j05HXNhroFq4E97dKRRGekJ6AxhGw847hPOlH4ydmHHYTrFPjPkI+5oo4m9vM83D3cHoqMglMly3MqyPGCvi291ipzzBfleHcwAcO9+PTb5yesz69FMy1/asilx2PRpjL5lOG5jjNGYQ2GA4SS0vCyFi4vIwm2VzwBZlfETAYzBOp50R/zxT5hpAYCgxXq1qEohEr+jQdTyO2QbvnucMCOuKlUUz+rR3l7Yt+XMW5gdZzC+gmNHVYfo1PMIiVVfxBfBcJXyh7rpUOtFBulKqOrb1JIAgmMAz7l0wtMk+x2L/XN+/TJMIloWaGA4H24UWy867HWGbwUzbBdho0jtgslImu0GZwACPxVw0wR6DwuvOGvssFqaJ9Uw2htCFCfjPOSL0IgAoulfkpGgprchBW13g03IAGrGXtjxsUAhbEmWyeYNY28iBlLYIUkC9gt7fDlMcRlfRMIIlFLl5Q5eNzvoybRttNOiZPQ/ypA+pK8bcuB4rjjxoxLE04XC2RAVSH2MUcoBo2NiFXvjpefLzD4vHJSwUDYnIHTQN41Gal9AjTsSKKxb4K84VqwBHXgVW6AkO66ayFagXT/gcYY/yecHbDQE40SvN8/BUNj1nCCBWHwngJIuC9+UxtgHpJlMp2XcA4gZtQEBV8S6HR7aAqc88tKUEFEVqfjHg9wnh4vaATFCt0+kIazpiTdwFBA8BaQeoqzNlevKP/0WDgk5mDenzh4ZlmMbEbYgdrU2ii6icAJq+p5fRyLOT4gQri+c7E8c01b0enJGF3LahlJrHllbVglf7TNOc4FP1fAL/Ww1Nq2tPvapDTbMGzhujXa/WJh817eJj7Ug7bldrWsO6a7fe9rlfPabEyjYOr9S37cLHcccd0XfH+YvM0VX7/MPx+PL4nr+bNhq1rUtrcEGrh0qfHp6NjupG80Zp9XbMcdtwb98U+reXlJ6Njlmz3zLfc+19oXri5bRGmw3qBf39aMS2zvO3uj+YjM2GvXN7Z9WdXat3NGnuZlraDtPO82897yhzvmXdK+eGoh2ZGeukWJs0b1RLcaaj82JxWM8UJq2r0qllueRiMM2Rdu8+r/e80ybHmnXWPpkcYn/qn43a7avLemOivTtz2x+N9zs7W1bxoniV5Yr55t2tNs4Dz7faSVE7nmhD6/68szW67pD61Z1qFvT7k9x5a5ofVbU399Ubt+FmaeusVleuR+9ynSIzq2/rrcbxUKNbu+O62meZfnGr92FydTNpeePD5vsauzHrdYtvnerXtl3Ml2pHk+puv5Q7Pm52ss1rzRq28zfVsxK/aJJWqV6ttpvZQyt3vvNRn/a0JsT0w5sd7ayJNXJcs7XWff3UuuZWofrOOj1tH1YH9CxPGtWrWrWhU8Xte47LIDfc6/ph5j4z6Jg1k/enb1jLwA2/ZSonw2b9pFA1tNsPH1zM/c710DAwLanmfSn3nt7cFtyhVzh1PtY61GsOx0fNbOeyk23UVb16Zl5stWzHbeYa/iSPrdvCLr0mnRPbvWTVVpsYxx4ZXd42a8PMZcMbdDp3ebVweelPNNAohYKbQJ5MBGmdEEfjH/BvXv3YcFyYCxYlGdxfyrL8BMV2WLO/A6+nb4T6wYVaMLqFUyXwhvRgOkqGw93qdSeU4IUjyhfIouFPrPnQHgQLMc2KERBPMOWI4TG1MHc8GTi7PQd7hjzxKCcXAPySC15gbMRrcacGA0lSWhp1xW0aSLmgQwIzcXJ23bqxzyNDmFs3tj5sI1VRlGDsgOYLE0gyAG3xcpfmWWmhnICrsw4mLiAl9Bo1MLWhsXEHCeJXQWeDQYPBzAbdmILPCDbEOL617LvoZvATd4JitJ9dCq7e+6wPSVJlPzyh9ylzR9FJIwVncs+5k2KZRMc3PAyPblAi2Lwqd/WklSq/2FyoDRRP0608XJtgpYpp4zuZsTsxZ3Ni05484xm+dcRy8s/9nciqJWaJ2ZCVWF0WS117bK8uSxvIQ3rq+drD5etIMP31nVrcC2feTPglUreMHo/V14RmHpJ4TywAWAI5rCaS/SDxmdUe3P2mEmiODg+kXxORjYnfJRSczgfSEnwso19uRw7fWyILF/bQBtyFYSDId+inffgcubeyGuwfKboD4jFif9vgfst6G2MYzECTzZJ7hz08TH6nPPs1ETr+2Rm32PByuSetAJfYZIl+yJAq0PkzBZQs5FA19d2yOcADB89IC2ocSPNfTkw9a2Z7u/lSLpPJKbt5XCgoekYxYZTOkpK6bnjcLy5RWsZbegOpFmAYFOeROZAOCGckpu1gnlWTuW2US/1iLVL+E8FaQWWx1haKWDVVvKtk9EyuoCslRclnMwWzmNNzWC3pUuWvru6YivqhnPrpQtj+3GJ5qjl9024SQubP7inhthfsLOvdA9rHB+pDl6b3AUfUpwbUNsyLaHHJIG4fvqgA/mWOfv7bIzXzVKgeO3+XfxyXKoDFnhH2zRpMPX32b9bsC1bpP/pY/57JkEGAcIdfmhIP0fv2Zhznd7vQQX68YH8brPB4OXwhklP/BUhOfV501J9I7ieS+8o8+7yWr/5Ecl+B5IiuFIy8ulvcLRZzRZWUstlizlALar6kArBT/1lILkswINc8MbJKKZfdJT1DJ4XsbiZbVBQFb4T5J5L7uyM59csAhvoTyf1dHf0Tyf2QSO67Hut/fyT3Nwv1t0EK3zOsahBW/4vj+pK1Pqc06Ljyf6THF/o=</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_4f82ceeefe1c4ecd9cc4f9e1f20d1deb\")) .filter((elt) => !elt.dataset['step0']) )[0]; root.dataset['step0'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtfet620aS6H89BaJcQEYkBfBOUdJ8viQTz8aJY0+SyWj1ySDQlGCDAAOAMhWN/u++x54HOK9wHmWf5FT1BehuNEhKdmZ2v9iTsYnuqurq6uruqurbcRBeW1l+E5GT/SDMlpF3c2TFSUz2rTA42Z8n6UVA5iRNSXAxHHndedcbO67v9oe+M3GcQc8dzkd9v+91J/7+6XG29GL4G+mddrw09W6uw98u/CTOvTAmqXVrvbsKc9IGOJ9gQenCi6bWnWUC7oTxPAGUOaS0594ijIC3RRInFHtq+UmUpEfWpx79M7UWXnoZxu1ZkufJ4shyOt0BWUzVEpcp2VxcGC9X+Vl+swSBpF58SfbPgYVrkuah70VtLwovY+AiDIIIKM3DKCfAwyVQyyCfNNymlUBRYX7TcDqD5r0LO7pKrqmgqqTvRy9eLWYkBYJxkjeO5om/yppAdpakAUnbqReEq+zI6i3X70eS/aZMI3nRJiP6Z8qLO7Lc5drKkigMyqwNpXYygCRppuvLptajLOThEnAURZ5ayyQL8zCBZvNmwMMqh7SZ57+9TJNVHLQ5y7QgE8OzCGCBihcEYXzJ9Mq/QrJhDC3UJtckzjNR2LswyK+OoPXyNjIHWVMLOZtHybsj6zrMwhkqTrVav7XDOCBrKNlxnM21nCXrHWuZrNvZlRdg0Q79H1aLVqjFE7qQwKturlDB12QDW34U+m8DL/fu02JR4qFELxYky7xLImmP6NF3x4dsLDnOU0IyP1mSdrqK21ckhbTMT8NlblHdtL3lEnjwUAKHiZ+TvJ0BjrewT/fwDxSb5ZbgwjqxGo2mdXJq3e5ZFvw3X8U+oloByUgaQi//jfwI0hg3sNMAgGWlJF+lsUVTHyGdzjxNFg0vT2YA1LIaC0pw0fGTgLxAUT7KG06zOQXsu736Yr4GMeS9blkQY3V2k5MM+HxQeYLIHGkjlZi8s3hBlFaDku/MVnMY2zkKryDD2cb1s/ifwjMt5l4cM5YjkltPUJkW3vLlnx8/Bc2c6rW5JPkTUMYwXiWrjAI3rr1oRVpMDQET0UQNkeLMy8gF7Q0tK5nPM5IzPsK5xVCt4xPLERiWBA/VcaY8lWGWKXcWiTIiETk9sdwaIjJnnYjEl/mV1ba6FdJuRyUuiDER+3lWUGRFfmk1zKTd5tTEx3Mvv+qA3EFmBbFmhYuynM8tl/MjtXSqVeisLOL8zDlHplxggZFrWgecvFWHZB1YLkeUW4cVdrmpMPehhbnmwmabCus+tLCuXhjX/7O0ZV22rNm5udPexGA6+Y9SPwvjq5cEqDd4eW/JDR3zf+JqH4XL5x7oduq9ex7G7F/85iT+7C2FWhbUs9yD+ewV2icBL6IBGLm3KjUYNfuTMPs6jGFiaNCsf/yDqRBMVY110zpEBOvYckm7X+IVFVwLzdK0uQCgtDKwzJDYl5TYlwUM/qEAUXLZqJZ6wLF/TXNoFP61TN411gygZXWbzUK37yQtLvq+IkfrRBkDMJ/JU8vQxI8aw+RfW1kdnpE1caZDFrVWMsQoSltZQC28dUO0O2eoOa2p6XEBUXD5L/qHdTtoLewz1sme1vJkvWwUKqDKALphqd6gOwWYqFrR9EUZICpR0iHryKpYDjXBsckDBAtjLeCaukzR9bB3TCXVEVia6hTEoMe4jqoCxfCKXX1JcJRFbYd5d1NnPeQ0BS3eswQR42zy3q1VKr2QZPlpUlWdEMo1IHGyCGOwMdJCh8O4IamAqdra0MdFQFmQRrvWFiqioyC60mwKU1rbqQwbG5ALf6dmk8iVU2xlFsiTV2hIv8pTsLjZXK/absUkIqYuZYp5nV7OGp/dpnfWZ7eX+NfsrvnaON2gfZ96GZhVlw8rUYJAdyQGr+YGIDpdtzuE/pnCEN0ZuYMu/L7E386oi79n5SBVop1abndcyp5XxqYOkG1UaQFCnRhbEahWUfDPoxdeDg5TDDYQtAf8d9MCN5EmFRMlNHID1TSkBh/8cyxAuJEFaQcHTc1CS5N3AM8Bz8JzoSAFuTeM3BsgB7AFqTcyKT75JO/O3pzLqVBIvu4g/y+JnzfQungDvMM/YctyW5LNV2rkXUW1GKNBeBnm1Hh+kYYLL8WmOqOw9qdz+sduwU93PhrN+vTnfD6aO4T+7Pqe0/Xpz2DYHXXH9OekPxzNAvpz7A+G/Znd4gRJbzTyuzRn5s+CLvvpjmbEn9sAQ8Wk8/WKQEqgcjaa4/8otkf8ERlzzmazEedhHMzHHk+djCdD+tMfzJxgwH72J/6kX3A2H82GAWMnmAWzMWN/QgKPDArO9grufBJFr8CLApZGU5ahOS3gmczDy4rPEsCI831MngC+GOGo7kEjU7elZXEXJsxgZAuD0pdhBFvFgC4UghtnFFrSQq4gtAsDl3byNvKvGoPB5xg0aNrTPYMiQVHQEUeUG/YD/2tON9McORrNSsfidNEILmizj7NCT8+cllX+d95SMlya6lYzPgjGebPix6lC72CEAqYBqLJfOJy23uV9bvYr1kuFlIBSZpsw+877jjmPTbmjVyQuD3z3bT4DOZwUuoMBlQf825QoP6gRQcSuJGL3XJ5/a5ulaBi30pSbserKOm9qJnjpmkMbPoN2AEfmRhE0a0FsKpCLIbAA/njLqrRdVaSmafpDNdfGmfn9W46J2L1/8zwsy1jW5pZrP6jpnD9E05USNTVCXffZ3B83d1V3S6fTm4hPXAE6vnwO40PeVLV3BNwpRsE0u+eh/fS+zX3fBn9wkz+40Td2sG2ZrtT/3Hth1meey1Zn2XuL5jy2nHs3p/PHa05nk9ydBzfnvckamlNuutJsEQ3cVNu28PHvb3sYC6yQUPLwj6oCBm0SnEoqpTTd3YP1RfOythmSlr30wFnIeXTYroTf3suiVHuVyD6jY+25bHZyEOBnBWzA1EqCj9bnh7c+9YmwJm5iklERtmnRuE1LCtzcvwl2Vk7qe8+SWr1EJ/aFl+bZ45unCFo45lQussOF8hudV1Og2botq2fOAcEOtkIMOUQf/+2BAm2FHHBIitF/CEYP/x28Hyb8O0R9Kb1POcYcxtfox4NA5x60ktxZmRX8ueVan2jxyNJ6KrDzdEW2aGFMLr08vCbFEuJxucIpYBbeJZjbq0BZgtBtNileg6u+BU4Hx2M6FDc72TIK84Zta6YeQxKLlccVxeI5JqMBQS+WCAtl6nhnCuFzeYxHKS9TukPiIiVL4uXZRTLH1epVFCnzuCHwp5CdWgcHoT7n8R6e5cBNy8rCgHAeOJeMZSkgWJEhAH5Ht+lw6QBsUwUuJEeH7mpldJ6qw4ohuMZEprFl7Saqmvl6U7k82HifUimoVqzRPqBjYcU6KGcQ3jjlLFJpLfl3rYWgGUOiPzU3Wy3VWbRiKtVNftK4fqZV78xs+5VOnjHDYBCe11TX0MBy/Y0Dj2wWycEuBvzR0viQloZxr8jOYthdvDVLKWhGsKW4R3HwLA5Cn2QNPZIdsnT8kYGeUFRtA5IY0M/4kEAHYbGWBEgWjAQmbKFZkHNmox1jn1NLBmYdknqRLSsbLaOzXGVXAoEyahtjT1WSusUuWC82zfBqnnFUbx1m9vm5OvEJ4BOLQ2Vvw+UFHYdsbalHYvf1Z7cG8LsjNZnEASS+NnvjvODj+5bL8ExUt+FYbVNZuDMJ16s2871Le7DG0xfNKgsiVfYo4j1qZDcWXvaWBFayypv2g9i8iJLk7WpZ4Vas31hffGF9wnHDyzhJ0UGko+WG1qnnq1odpqrZapblYKPRvluoIOPtgq5V2+eauyg4VVHrPMcKh6v4bZy8ixX2aqwGCU8urG5e2kX22AVNokfrztxtO4hi7rOn27tOQfMhPaCqV/dqNlakxvqurba1zTb3kJr2uqtzROTijv90amtORRKRDknTJG3YPzJe5LHf5vOIcWcX3wXACniThLHwPZQNpo+gkV8tiV9Zpb0A48+7+THOw+gntuG7ERCM/dHtyS3Lo2BCdtL8RbeHhzDr3Hw/y0h6TTfv8H2wJM0IxRNZjQbYuGlIsmIjs2gvnn7mnHdCCfElFg8K6FRnH74V+7mXvsVt9yeWxG/n1xVJb16BPevnSfooihq2vnVbFj2rXENelhCuEImww2iFqSoEIJ2ULJJr0miaVLkqoU4QZlCJGG0PvTFb1u1dsVcYqpHlj2JwHJDBr1NvQaRN4DXEE/ZDbj9hyph3ds9WYRQ84vvMvw4vV6nW+D6Nlohab1MVlcGLXamrLMqqeU/+hPE0T1gQGwM9+CXvdqW76h97GRn2SyApsQL7lIWKFFCaJkPSees5TE06ZS1DxsETCBg0COg2dA4vJcqwa9F9JdAyTYa8MUDeGCGzCOaAwACuZcg4ahStRPG1nSCSlRuuYSR8QVLcBlIiKMmaJFfka2pj4/rDM8kAVqRaBzTdK8cp7Ma0vcrtVnKTwmw5Z8cMpJmSArAxRT/0IGlJdReFRjiMt5JlpxJqiQrM/Ap9ExxXv2LzwyrOVstlkuZgBgV05m9Wt6tTvbtAY0ktlJ0T0bSyKQutkFwSoZPLt8Jjgr9KUxit1cSMLKkT48hejBZKKvbqlirbEcGHGz2pqe0xwwLYvMnLL2P/BT/I6oH4Lnb2U/6VxDu9nmqN8yT3oidJlGn1TqKf8aAUrad7Xmaw6oBUK26cJoBKvY376DIAAJwSWAqd8Tgm7siFnA5YT2BT0Z/UxBJgJUdMZKVpItet/P0lkDygIoaycKd/rByjKGrOqBV4isIVknuZvNMkB5r7DQkvr/KK6G52Fd3NfUR38x6iu9ksOl658vcW0ZVVl2SHiM0aVeRTjo9xtVc4An9PrVss9PauXj7aYL2DkDQMSVLVws+onGKwPs6lM0B7Ote+B55bJi/dNYLEXy2g43X8lHg5+Soi+NWwGahdHKOinx16EBH3fpeqeWB18XiE2H2ogF9RyRbwtD3M8JJQn1BcXCcl61zhlVPlq6iQ27C7AWXRMEjwHdf0bMq/8Y3ZSpy4jNmD+YnTGzahahpJqyBi5nxZboCXQ86blq6M+/OMKJxn1Xrl+/yhwAr8QjoOIG3ur+wO57v79Z3hlfVltieGngB5NMtMJRaZ6saEAtFbb0CkmRWGzQ11wuRbXcuoNAPf0E63HKNAcMs9bt7WhWNev+cGJN0A/1yQ3HC8qfxj4ruFpbZKEbZKobQsp+MOmtNdq6PwhIkHeIDvsDyXtzUYztfL8JCQQXXC2NCE99EzdvhoIR/U2VYppzPYoUU2tHAbq0NbGDllX1uCMHuG0x1gvD/TT1BKYV06AINPqY3AynmLgsTBiewhyAPxl5sH6qnOIXchyvlEMkdQA9nY5Gh7Km5K+JvN8HwYK6d+C49VRwRG5xQPLd1K6yfcKmCwRWotuPJTchAAnxrOZ4W8zqtKJwKNJ4o1zAcA6090/cY6sj75pMyuoWfYyS5HELS55R473Pcq63aKjio6aP7JjKng2ot98iRZxbmsew81qrhFJHQL7JsDqXELAwdTSzuHggmDSIWt1WrFYFPVtxwRZD5OTyRDDZeLK+uw2qcqm2o11GpqslN5b2/lHbqmwp1OC2UGVBo1Qmtuxq9ZeNY+Z2Bvva1fOr7ThltNd04KcRj3I2ilVcqS5w1rr7prQxkH1vVq+xAnivo/G1QWh6iDk9KnqdXXWm1db9ZWFOda1dX1Jl1VPtZb9HS9SUtrdXRt1NF1vY6hkFBDzVJqbkI2qufO6qKYr2tdKdeblHKvtoS6yVr9xzCEd6IwJj9zp8SdbgDM8jR5S2oW8+soP/GWCJz9uvJSshX6Lwk1tewFLvDav+sUu7dxYuOVNe05YQvpxVGxNppkLcPotw2myP1S1kH33PrTn9BMxb0JGzCkcbUOpW5KNc2jbu086n6cRz/Ooxvm0dMPN4/u7TZ5ujWTp/tx8vxDT56nH2DypH/LwbDyRhWSK3GKRkzeid/qfiUpAyd0U5CjyZeyBQt1kbaS0j2jZLjlZHtkTI7cVRZp9YhgIQm2NeEFLmvhfQc3dcccqJ8s3UhEgw80ACMmMrzlw50aTtjLWDc1WKpAcJ8zXl+D+Pwnpp7KHbmM8Zf3KJ0D5J463J4qU5QU3ZawDLfhyLHMzVaYODiXraJcinl/sMiKWBxmRMr4P3TQncIpFudNDYlTetN7hm8UpDs1ioLajvn8vqnKXQqhGr8ppY/j/ClzltrtasU3LS4VMJRZa/swz4N6SbCKVpkKT29/KnDol4yn16/8/FzQm1aOuyZpoF3jJSMecr4r2/TLe8iQwIFhzUd0FMzn/YMhnZaLRepwaVBrefz9/VUEep4sQvg0qsjNw1XkZicV2Wqv6joiI2xUkmoNH6YkCuIfR0nEPWt6fLRlGcOcLc5OGbQ837hAWV6JifF86Z6gcgsWhi7j4Ak4nEHtMmAQXttN9S7FMKZUpXU5fUWFFXx/+pSygsZW/MQCX0maXuBZrEba0sWo9tQIWqxE7gCLVwN/TW8Gpt68uBu4BjpPvTjDzebfp+ElCwDkyRLGgHkdfZi4XqTJkqT5TcMOF94laacEtT+ML/GKF7rnBoQU2M0dCLTb4gLS9m9JskAC7o6IeNtYm16EnIFlQjH7y7UtxM2aQxX1a9+L/Ma1lza0ctFq/uxWXia+W67FqUCZUtESu5Fi4DW0xPW09CABygzMA1soC8PfuZFkcEVO9J5kFI6JVbu6+KwLnd/x+y2UVdT6s9vKwH9HL3DsDMgCx1mor6hwDb2/JkuJ3HoncpVdDFH0rTcjkby5g68q0fQX2tEHPrfg+vsC+gWFoSE1unQe4aeyfk5ThAbh5VevsCOh9JdSPE2GYjclP6a3bSOc03GhBtKdyUYs1GF29ZE0FNlUNHhtm0uXZRtOZ4iDa7URm6hxLEPtEk1xHOXu998zROujnarePGqaJCH3CHFls1FkkayO9Z2Qey1S/9Pbi7cUI+Q6zuegb5/dhqh/VP3MeHw8kWq7jRPddC0jlAbmMAr6hA3ywomgGSZWEPYR3sCOMpNGBMuo6FzLC4+2MmWp+UUPY7uL5My7330v1T9ZpZRRM01yII/DZnviBOTSNtI2jMtco1Ic9o3FpMr8sbPGmTS1bT1c8++twVXDeqMKG5RTlckO2rmnWtCgqt8+WCVK9K29SwLdQYUq0LNy9K8HElqwCQbGfrE2ZGN710MqqlsoZTuhWolKrGilpspy7y8pP3SIMFCQJmMGc0uBjqR6tPCkFSZR2nfNuh2ANPDyOFmTrN6Cfy8foSyg40deln0bZnkHLBawdON5gqLkzzAUltN9wkMcjN1yuHGbKPeC2K7iV5wpTfG1lQSJ9V2qjc+S2NItzlpZnEbj9b/Hn92WfeTu7LW2gQcfwKiwVlcofTlD6pUMmUdRLZs9pmFruWIrTlVKHIDtEavP9taquF0NIMvJUl7bMIuCiZOhbJOafW7ZmpyY3jxQTgy5kBN9F8XWMuvFxAHqxCSya8XEAXQxFclVDwvt1b7jX+HtKhmfIMoZYpN8GdGq7HwMnELOExhv8KES+lwFuc4Vg70iCwDoQFUvSc6SytCHplv1gHvqSnX9tmV+xYROSeneUlyoJv6vNzmMPV/hxQ04EBEYLoVutExC0fvVQ5Crgy5GcSKg8AGHXE6xHH9ESodupLML+2PBDsTZlbPreNU3DLV4kB8vtekM6Jl6Fy+hkVcI1Bp8v+TTuMJ/UfYurCeUhDxwqsQLjSruDZfPy6A5VR0IdhwGSmTjQCBls55uu4Yc2sntriGLd2/bi2+qmaJWhgNVMgmSP8rzNJzBZN6waQu35KbVAnHzBF/a+nABPk7ROHNrIJudegElRvO/J/h4VlsnwrNLAZjz7QPqz9OTyfOkWY1jFM801XawqhG2k0AEYSaR72CgwnqKVHuq86BN5B+OBxyMqjzgfVga0P3CdVX3QyO0c7SuSolbYMUjTh8wDi1o1pmYBYAanpOuWV8/p++6wcShjXQYAMKNUzTWj+0F89BjfNIrjC+fRCEw81I5FCwt+cTgFr0U4vrsVlDirkq7IE1jLyCo18YVIREDkGz+mp0ghYkrwnQSjrrEw0E61EfQHKiCawO8HGs1u0gobfmgNtaC2rlchhqpLcJkSzgFvtC+01J0LEW7fGsbu5qdr3LPAhUNGqRoWjQLg++NttMZoP3ldrpkoV0S8T41tFQ94fHbirKUQuDaggen/Cv5nj3DXpQHk3bUwIm+q0TR8/kqEpUXtDdpcunfbtNjTrRAwO08W+UpcaOemprnR+XLHQKKVrklCqNfyoMcebI0YUFyiQQfCg6V61F5erbAohklHv1UMMWrklVUllPism8J+c7wPsumZQdJD2jot20pIpGHo00LDpVwF3sQk0bgigJwaUWiD59KfKuGvjR0qhKUh86UrwZtI1asIMjUeBBHIsdSSnp32ql0nJiAGpovdMtQVu5rYVq7WsI0QzD/6zRZvOKWqfkYobI7AG87CV6s2fUZfD2cpjbehXGQvOsE5Bo8DFoqBVIu56+BwdeulM09ejGudSgXhZ/bilPf/1kWAI+CN6ssX7AYoFZQLdX6F3f8LHux/it1++gjJWnGzvM3dBtakYOMpdVdJejq1UFsBf1LrRJN63PpSY2TygUn77eka74C8140vVWe2DXNRNcsUQ/lGsqb0L40tmRdp9qy4Ax9jBYoXVwlW1jKEhrj752X+1eS3uo9hm9KIUGoXjTM1WqB6M8xt/G6kRIaUKavlH52W6N4d8EShyAtLBJxt75SvsXK5rfGaCEA/4o6ja0CXZqNTOOBPFtpFZfiFuIHK7gaeKgr9a7iRNYHLdRKbhjcigvkooCtM0vk2W1OkCxAN9d5T2m5zE+TKHqs2GS3dNI1lYDPGVIOWtaMXHnXIT7kauMVS16c23d6GbIjTct5FufJTyF517g1oAPNKPHfQkpMPFCikuCdeui9Ks9FssqoZqBM9RiauPwI95mC5OQNpxjaYpX7W8sqP35RxjSBaTgBXbijrFvyx3pp4DeJiXSDpObF1QHWXQZdnIa8uNauSSw2HxU126VQKuh69thatGKlb3E36aZZeiC6Zg2tUkauWS/bi6AHPrYXgc2hLoiZb55Uyq4X8IYrc5rTHRRBF7UUnADTCCCGeh5aTCQQC2UFZaaaLPmgIGBG/kZECTRsnm5Cp1eKld0BerrMx6nlWF98oYhMBj7QgJnrJXGsOoiatPjWB1u571aD4d6x5jnutnSsVktUXdMdg/NmZNPIwU5FH9QVXVdZRSJ3NS31S9FS3wjnfENT/VI0VQEtt9U3Bm9e4432281tJdaL37OxfnmfxqoOL/doq1/u0Vbl4rhdewRkp+krYRaBPnvtOMXsNMFsZYTG6bbNoS/FUYMHzqQCvzqdVk/FIMDDJ8eLclZkhU71G1Uvrqs3wG7mwnyzRxEAVWce+wlmkOAIg+WaNm65D1mvTOW6E9Of3SYrS30xy3ykSD1RtU0mNQexKmj6uwDmG0J20tV3V4REJl0V46MX5VBuNbAVkCj3fkEXg+E2Oyyl5Aqx+fXyT8ncA8XRQ6LlW9QVo72Jby+U+Yy29u7NRktflkkVEPe1a+xy1wBmDYT7FviOdnTZi/vpZESF0FQ/CF3l6NTqb6td+8Tq6/qnlHpcwy7eZlO5zkZlV/k8UE8wmo761dbj2GpvrcjBtoqc1lUkjO9Vkfb2ihgWM2USu/vA1R8PW3qpOpW3eiSXdkyf4v4ix/0whGsdajr8pTbmNTQJKdBKHJcFm6XC/qYv93zI0ky+bNlehutql16WhdfkiD3gcqesiZl2UWxqQGMAo3JhLQtUfR+Tp/zZng93Xa16u5B80ahh8ysLXnAgkWKAK8LFMihLnKrXM1Wv0Nv9Er3KNXrqJXjVa/Mq+YyFXN8tVn8xXvVeJPqgBHu3gU/+eNcEaAluiTiSHloCzeGzrnYytjhY9IBdpLVHeFzfW9p1UOXpnY1gCxpxZG1uO53RgCxqYSWLNYzx9o626jlX1hXMR03MwGgczyN6GNi+ZipdC7v5tJFBYrXnKHRIrFXhmdvdzsDeKb67LXxdV+dVjgXSRlqutSMbe5vOhhWHvJROsFE/ZJA65ZBhajYg72mLzn/FaaFyjTnmfBVVfPEdNb6u1oKsbBzxJL3uXXnvcgXMePjFwqA8dgFwNYvlTJfu84KZSFRW3DRwRxava+lv3LxdgRbBlfZgE8+lF+u6m+CU7e8+ieWLdapFK/rumgBVd0mfEO50fWAzwB9HJVh9/2VawWM9/3KVEBO/qhV1W5aKtqze4c/P+N7yxwI0m+iomsTsOMN1+kemRIS+m+7dNakVll+F1AR4mST5d0lAGs3OVZLl4G3O46wjAlDiRkb4OT0+BMs5XOanx4d5SkjmwxTQTldx+4qk5PQYN7hbdIfVyf48iQJ8zeMiBsr7p8dUTqfHdGHJQsvhZN+/Iv5bqMO+EeciTy4vI0Q9pEgqeXrbx4U3m4EXvH/6RZRP5Ww7TnKaaZ++8dYdKgQLmAcIhQz66HEJKED45fGNfgs8xxq61n//53+0nY4zdq3/93+dzmBonf33f/4fSJn0WpD3X05nPD634iT+jaTJkTvcUrrIFv9wcclVptBkDSkBCfbVPDwwDhoZXPioYaAlG/MhE7eB8jzRuheFXmrI2jMb+6cvxVTPNKPT6Qi+jVpBVYa3OfSDKPRpnzhM/Jzk7QxwvMX+afF2FutXVDnZF+rmtIiNzOl7ARt1d8rAOsDDqySJxfsanES+5NEHHHaVV0Uadk4WS7RbkA5JUxAYWIh48qS0ZdnzCH959f13HepJN5Bghx8Q1+mxuttNZdAAaohSPDBiqR2uY3ocg7FbvqyBg8aW3ijqIppRqdL+hnZ5kyWgP7f7uDly/8jaf8rib8UNO+A+YNQQHGbLozsBmh3rG7RgD1k67ulizyLQiON+y9qXHkNAitF19uzFu+jRwbfDX8bPuwfRM59cj7/JRunwJv312cHwz3/95sWi9/zJ6ODHR9Hl4d++/ml17b51vp1NfvUGP0x+fvqDfz387afHT0fZVffR4d+/vfn2xeXJSVEUfa8BS+JdGTO0txEw99EPXz2u/p/RkS5nANDbfc+Ff1wswIEffXB29svDuZBydruP5xKQLIBCLu3C9BOgLBdwMJUeSoBU8Kb2oRtRUufw+6aOllOl5WyhpZ2GQoKYrN6QRCslxFTepYR0YZ6HRDqWOZPxZDJ2hpPRsOeOMc9bY4mmLB6nhWzmIO4Xt1pTDGfU7Y+cgeM6w0F30O+6HIJedl1HUzz8jHU4mwzZeYMefZHW7fXZZ9+ln8MR5Azx//RzDD9H+LYu+5wANdfBRwDZw70Ofvcm8JfL3unF4twRvjQ5YAldSOg6MJq7E/Yibh9PPmAJ3R6DcLp49oH+xQoZIvoEHyZ0HJrg4gmJkUNTMWE0QJaxpDHF6OHPbq/4RmKTfgGOJQ6x0kP65m4PqPWRouuen6MCKmcDQEQjpuZ1IXamWaLN+bOC+wKHqhd2nn0MAW0EO1NB6HUltO+BypVKWz4UhpopEoSibisDfm8pxH3/Qs631/XfYwvGexWOZd6d38lDMB9u4SdMrpUJvTL7aybMpqn+H9ZTujvpyHry4kfL4cS+uMyn5lIYAzjFU6MWhn4WuzhCe4TsW2GAtlZ6UcwIc783783Gg0nfdfvOeOANh47vOnMYJntk0hXWBvyN9E7Zq0BgLyzQWblQuD0CE6nRUSy5oysva5wyi6ZjsvEozhE1B0mAdys06WQdYcTyU4/+gQkPakdL/33MTIsXv4u5acgyysLUqgAeecsMM6nValAMjcanRsv1f62Fu1Ei9Sr9OxvEVICNszPc194bOOPxxGrhDNgf98fDHj43ix9Ot9sddlnO0J10B8PusHzK9gxBhsPhwHFArkCp545GvQlOOvgxnvT77qhHkQf9IZTh9CVkABn3e6PxBKc1PvdOIIMVNhoOxvSjnEgtteSh6wD1EQNxu5NRl06QSHY0ckZ9jtwddScwXY3PYUJhXirtJVwhmjXSf7/hpUc8GGAGJIAq9XtjMgt8MuyN3d7IcRyPSMPLH8+B7A5Y7+qz3uV2XNq53I7z0Xv86D1+9B4N3uPhE+8v179FPwLKVy8OfpgM37lfXV8/SZzZ4Kn/+ODQG/iz6+7s6i9PR3n31cHh35eLRy96Pz9zUmd1/cPzQzJ6troeZD/+lvZ/vXGGB7O3L92P3uOH9h5hBBu5A5gxnfHImYxgyim8R2PWFu/RdbqjyaTX64FjOJl0cWpTvEcjzY/e40fv8aP3+Ht7j8R3hsGgOx6BodkfdQn00lE/AEN5MIGOMuj+vubdR8ftn2FZ/s/22srJhrlLk9Fo3B8NmfsD412/PxkxF86duP3BpNfXvDbwSyZ0QEZHCsZuZzjijpczGo0mXe5VoY8FjpvmeI2HExhxYRjGScjpD8GxYchizqLIAxd8PmgMHRkmBkc4m04XfEecp7D5HHcCtMbMa+tNBu54PHhvr+29TWx20c6zgN7yWVqFw5HXnXe9seP6UE/fmTjOoOcO56O+3/e6E5/aRDuHm2Tgrc6jDLx1KDqfck8g8VMw/o2+gDDXUVYciINrBvp+pxDnBWbvV1CfRAnd4SK+Oz4m0GLogz3TMiclMMj65Ocwv2oo6AXReepd8uOz2raBp/zzaw6BdRDQytKzIIz5hpP3ReNqJ+/ZU8DiaGxBGT75hoXHN8+AuMDGNzGm7BHpZJX65Cm9c6ZGhp/iXLJvHVgaOr/RSRZLQa0zD9NMlE1rBghlbukb3aktYRTyNlfo/wM9JZdd</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_4f82ceeefe1c4ecd9cc4f9e1f20d1deb\")) .filter((elt) => !elt.dataset['step1']) )[0]; root.dataset['step1'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "_, state = nnx.split(model)\n",
    "nnx.display(state)\n",
    "\n",
    "checkpointer = ocp.StandardCheckpointer()\n",
    "checkpointer.save(ckpt_dir / 'state', state)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<script> (()=>{ if (customElements.get('treescope-container') === undefined) { class TreescopeContainer extends HTMLElement { constructor() { super(); this.attachShadow({mode: \"open\"}); this.defns = {}; this.state = {}; } } customElements.define(\"treescope-container\", TreescopeContainer); } if (customElements.get('treescope-run-here') === undefined) { class RunHere extends HTMLElement { constructor() { super() } connectedCallback() { const run = child => { const fn = new Function(child.textContent); child.textContent = \"\"; fn.call(this); this.remove(); }; const child = this.querySelector(\"script\"); if (child) { run(child); } else { new MutationObserver(()=>{ run(this.querySelector(\"script\")); }).observe(this, {childList: true}); } } } customElements.define(\"treescope-run-here\", RunHere); } })(); </script> <treescope-container class=\"treescope_out_5ecb69e45849446495cac4e3367e5c4f\" ></treescope-container> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_5ecb69e45849446495cac4e3367e5c4f\")) .filter((elt) => !elt.dataset.setup) )[0]; root.dataset.setup = 1; const msg = document.createElement(\"span\"); msg.style = \"color: #cccccc; font-family: monospace;\"; msg.textContent = \"(Loading...)\"; root.state.loadingMsg = msg; root.shadowRoot.appendChild(msg); root.state.chain = new Promise((resolve, reject) => { const observer = new IntersectionObserver((entries) => { for (const entry of entries) { if (entry.isIntersecting) { resolve(); observer.disconnect(); return; } } }, {rootMargin: \"1000px\"}); window.setTimeout(() => { observer.observe(root); }, 0); }); root.state.deferring = false; const _insertNode = (node) => { for (let oldScript of node.querySelectorAll(\"script\")) { let newScript = document.createElement(\"script\"); newScript.type = oldScript.type; newScript.textContent = oldScript.textContent; oldScript.parentNode.replaceChild(newScript, oldScript); } if (root.state.loadingMsg) { root.state.loadingMsg.remove(); root.state.loadingMsg = null; } root.shadowRoot.appendChild(node); }; root.defns.insertContent = ((contentNode, compressed) => { if (compressed) { root.state.deferring = true; } if (root.state.deferring) { root.state.chain = (async () => { await root.state.chain; if (compressed) { const encoded = contentNode.textContent; const blob = new Blob([ Uint8Array.from(atob(encoded), (m) => m.codePointAt(0)) ]); const reader = blob.stream().pipeThrough( new DecompressionStream(\"deflate\") ).pipeThrough( new TextDecoderStream(\"utf-8\") ).getReader(); const parts = []; while (true) { const step = await reader.read(); if (step.done) { break; } parts.push(step.value); } const tpl = document.createElement('template'); tpl.innerHTML = parts.join(\"\"); _insertNode(tpl.content); } else { _insertNode(contentNode.content); } })(); } else { _insertNode(contentNode.content); } }); </script></treescope-run-here><div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtWgtT27oS/itqOnNILsRJnBcJj7lOyIsWKIQWyjlncmVbdkQc29hKQjjDf78rOe+YFFoK5TQw0xBppV3tS9+uuuuzoUX2JeYR4muOS1qe4zD0D3IdnzLq2EXkEQsz2ic7yHBsFjdwl1rDIuo6tuO7WIPxQZsyEhdfisj1YMSiPouLreNs6MKo7dgwrGKtY3pOz9bjmmM5XjFYuoNG31QLCGA/qrN2ERmUAZnNiM12kIt1ndpm3CIGKyJZa3MmNom3CTXbMJKSsnwbm2EKMk+Wjf6I96lPVWpRBpLjHnMmtHFqM4/aPtXiPr0jwexI3PvdRKCe3Yl64l7PBp4ejPmaR12G+Pn2NrDrWlTDXGMJR2OEn94juLuxH43G9vZBocDPZ0gnhu2jPcTa1JdMws5A28eOTqIxqe34TBLzcDTCUMslNj+yovFd+aI//w6bqWNbtwhM2z3L2gk4SCBm03FsGI0OHK8TQ7MyOBcwxKfmhhnV+KBLPMPxutjWiGQ7g2hM2BcYRJdmUDxYtIvScgz2oQaKLkgtWcQ2WRvt7aEkJ1kpukdYz7NB74hYPpkK1u7ZXLLFrf02NRiXTxDwP+7h9wEOUfAqW3cGkkduesRnik27wlxVD3dJNNBJjO+xs8TI7fntQI07IWccs9gLjrHilI+XgUsRGJI5pmkFUdkSkQPe6vK9+Aix2BYifXDwkSW5dOK71CFDrvSIF+ECjYglzcK+/xGCc7RvNDLZs9UFN4yMmd/HQJ/g/sLH93cTYQGg0z4SG+5F5tNHBDGswknJ7V4kGYHQ9dgyiWODiKAMG6ZWBUO4BqJ8zfjsEQjGII2JPNLCquqRvvAfkVbe57ZlnEzCqUYEmtPtwsIZCix++OEXSHDRdli02Hb6xIuF0M+TtwZtYrfIrQsmJ7pYKhmOpWMVTmDD0Ypt7Ef3LawSa39+phWcM2CntYnWIXoshv4TQw9zBQ2CW+ozFEn4MYwphc+Tmzm7RTqVTWU5AWiXeB7RWy5kXdIGWYg3S5jnv6OkL9RbRJRhyHJ88ZzoD1wXoMuwEwK1Tn1gOhxfC4uEaB8JDRWLKoGMQ2ak0sTPTii/4GqIp/jdMLpCwOYTXtQWF4ZqOfyOeZCnsPQyZx17HZ9gE7zYXl79TJaeyMCXhi8a089JKO64Itr4S86q2sZrije/6EEhcy8gJLcjZ9zzfG5A14GbnnghfKn/fGxFKAhGcZGb/Id8/Hm4To/HyC1b5iJRv2VQz2ctx25x9w8JrVWhJMlZHk2hpkI/LH5g8UUR+am62DMBmwViiIC+/0FukA/dodpjDEBRWAKaToc5bQRFFqhAkQB4w4n/IqmMHlnAyxtHGLyCYgs1h13VsXx00mP8vDoqByvh0x1CYMQHRO0A9g0ybxfuvTbkcECmNoPlFPtEnyDm9yTJf3eW3TxYLSBtUiqQ7uIpg/gIOUV4upuulAbYb2lwD4BiJ+uxweZuj3GeXsVzYc08y1nVoz72ovG4jhmOYxsMK0BTbHaYM+FI0MP22JvFtijlIwIaA5gfd3rsaUeZSACGoUR/Ny+JYIne0a7reAzbS3urntMBMMBHpsno29qdWTajz7GZ7yWOq0AwvaUBKtc9Yo9Ena97YM95wmfDIpPQGV2kc6GqYUuLQnEGlUHKvRVgUvIZ5usn8v40SVSAQpB3A0l0h8HZuRSzyrvpYcsGrN2CKtWgt7DJXJhsizCBwgJzXDTAng2B1xon9rEtDANrqXQIoQvI/J9JpeqNClOevUZKGg3Fk5JIq9NyuCiKWezFTQ/rFMwWTaWzOjG3kAMebRKUBOlyWnsr8HBA1DxfiCE00vKSKEuJ9dkUP2fy8WHuJYF9QUkWdiFFfRvmPT2HP8whAHeCxxwEf4DmOeQIYzE66DxBC3D4tHQLU8vD5M9jsy200OCR5ss+9EiBF7U8MUUL1oEI7IeXveh5Hyn+nFFXG/IxXB+losfo8jHMvnWo5+hw8aYDeqd4Hh5Khud0o7qj9Xg1LPHk6kt9bPWIH43FJN/pkqhIubxdwT+lAB7xVsUjAVJkA8VQbNIc8tuEMN5BIgNUbjab/DRNPsb7QWJS8oiocJtDW4v+778jUKaRcfJ/OkCbLYlt3hGzRmODUTMyw5sNvqcVUc+zohwtFPl8YuAYhryjAh7JZbb0ZKF2ZColRfw0ThXFEX+Vzgbwb72qKBVl1U+pqyhmx/mgNyql8uCropx/LR8qR41SWamat436xzbzS0eUmOnqwaX8sZH72m+6PfrpKHueOrxsnH056l8c3bFPw2q1vHlhds5p6SDZpgenvcOKXrtO1tWE0W/o7s2HXPvmgtLT3pFda9eNz0z5nCsdexml2rA7lZz2udezN8+yN5rfGfSNqpW4uTUrzrapHg5q26m6krCVs+xHzztMnW2ad8kzPakcGinzOF8e1K5lM+kMe2f5fLeSyg3ql4UT03TJeWeYIQ31Lqup3kmNYcU8bRwPDrA/9E97jcblRaU6UD6duo2v+udEYtPMn+cv0yxpfPh0o/SzsOdH5TivHA2Urnl31tzsXTVJ5fJWNnLa3XHmrD7M9krKh7vStVt107R+Wq4kr3qfMs28bZQ+VurVo65CN7f7Fbltp9r5TfXL4PJ6UPf6B7XPZfvaqFRMtnmiXVlWPlsoHw5K2+1C5uio1kzXrhSz28hel04L7LxG6oVKqdSopQ/MzFniqzZUlRrY9MuHhHJawwo5KltK/a5yYl4xM1f6ZJ6cNA5KHXqaJdXSZblU1WjSbXuOa4NvuFeVg9RdqtM0ygZrDz/YdR1X/bqRPO7WKse5kq7cfPniYuY3r7q6jmlBNu4Kmc/0+ibndr3cifO13KRerds/rKWbF810tSJrpVPjfLNuOW4tU/UHWWze5LbpFWkeW+6FXao3iH7kkd7FTa3cTV1UvU6zeZuVcxcX/kABiWJIdINZdEO49QbHIf+DfybRj3XHBQw2DUnRw5YkaQXFVhCzf8Neq7uCbdFUFTA5QPCwN7iHraFoAKTnW94QgucOD18gGwFtPuZDeuBb8MqBw208wJQhG/epiZnjSbCzqzrY06WBRxk5hyI7Ot0LDjvaa9pXBfAXjcyUFbyjClzOaZdA/REdt9yX1nmkCzXC0tL7LSQnk0mB8SD5AtyLigI5nO9M7RCZCsdbA+MMxpvQEfQeVTG1ILExB3HidyKzAa6zAR9DNqagM4J1Xvpszupu1B3+Rl+Yl1HjxvB8j20RkUb2d4N7fZfabm9000TETa46t5HQTUaXPkwGFz4IIRbP852/aSP7f1iMiw0Uq+nmJheqhci+YeFbybZveU3DiEVVabxn8NHkw9F/dhOjU81utlS3RVbNL0zONnPhMO9v5fxOUDGkgi8jAYroYe0/q7JDz7gxLWM3kGOXuRvvbTwxjkVnP7aBJjX2XuTPjdFZN/6OIHHv7kVmivAi+uOm57CdGbJgYActNQ3gmheeDJmyDX+P1Lw/b8afazeVYv9FjPYqkdHHAKFAEn8hNL6MxoMQeSXv+XODK//RXjQmf0VvEvZ+SfdY8ZwVCXGQ6dx79KAXhdaVP+xRn7CHu5OhkIy7SvhniOrglTEIIDGBgqgOEVyIGsz+Ya4It3BhXzAwJO5AT4wOseYZQ2TOAyP7CNDKAbgEr8MwE30ylMtuI8dAic8+8fwE7Q/vQHtmIjlIJrj2E2oPjJZotYhOmfD7lsTH40mpICXj7jAd5+UvSDkM6MFaiam13OGCdCs+tpaNOOkqgOxrS69W4qqQE22DvRDXGL16R/aPwYiP0PDWL2wFccinmkEselY7BB+x1UBhJnO9ced/2fv/8S7/TezYIZ5NrDV6fCX0GKj/Cf4zXrBGkGsE+W9HkGNffxqymFu1RpFv5iJ9fWv/EJIMv5+ovheZ/LfIbaKmUljN5PF2MpPZTmO9kMxkk9ntgpohqZyxENJh/50ysn8NkSxexpBhOZil5WhmC2Vi3z7D8utdqJCGpqeS6XQ+l8mkM9taDhs52cjn5TTJpnAmk4vs/+zUH5Y3w3XB09wD+ljGlt9SzffD+5XY5GVj52nof37Zz82VX6gP1ym9EzuiNtUhohC10fQtmT8yf5f//maKfsZ6680Vua+Fl8M+7t/qXfsyTx7PWKQGIsm/0cOU/DgryeuHqXVrIdR7ntKYktcPU+u2wm/RVpC/47lCXj9MvUGY8wtYev0wNafRR1dM8vph6q3e/+uHqX8LenxKoS2vH6bWCPJ3QpBPfqqQ1w9Tb/cifX1r/+yHKVnOYbWQklM5OZ3JGakCTuVVdTtfkFN6IZnXf4mHqWxaxnpSy+t5PZXJZ7NqKl/Q8oR/z+XknL5+mPq1Hqbk73svkdcPU29V0b/pw9Sr4uVVD1NvTIkv8+ARqrDndM8JpU77+/8HnO2ZvA==</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_5ecb69e45849446495cac4e3367e5c4f\")) .filter((elt) => !elt.dataset['step0']) )[0]; root.dataset['step0'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>\n",
    "\n",
    "\n",
    "\n",
    "<div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtfet62ziy4H8/BduTbkqxJJO6y7I9Xy59yekknUm6p0+vjz+HIiGbCUWqScqR26P/O++x+wD7Cvso8yRbBYAkAIKU7CRn9vs66UtEoKpQKBSAqsLt2POvjSS9CcjJvucny8C5OTLCKCT7hu+d7M+j+MIjcxLHxLuYu55t9XqjYb/f64/doTMfduejUbdHBrbT7w/3T4+TpRPC/5HeaceJY+fm2v/jwo3C1PFDEhu3xocrPyVtgHMJFhQvnGBqbAwdcMcP5xGgzCGlPXcWfgC8LaIwothTw42CKD4y/uLQP1Nj4cSXftieRWkaLY4Mq9MdkMVULnEZk/ri/HC5Ss/SmyUIJHbCS7J/Dixckzj1XSdoO4F/GQIXvucFQGnuBykBHi6BWgL5pGE3jQiK8tObhtUZNO9c2NFVdE0FVSZ9N3rhajEjMRAMo7RxNI/cVdIEsrMo9kjcjh3PXyVHRm+5/jiS7DdlGslnbTKif6a8uCPDXq6NJAp8r8iqKbWTACSJE1Vf6lqPspD6S8CRFHlqLKPET/0Ims2ZAQ+rFNJmjvv+Mo5WodfmLNOCdAzPAoAFKo7n+eEl0yv3Csn6IbRQm1yTME2ywj74Xnp1BK2XtpE5yJoayNk8iD4cGdd+4s9QccrV+qPthx5ZQ8mWZdXXchatd6xltG4nV46HRVv0H6wWrVCLJ3QhgVddX6Gcr0kNW27gu+89J3Xu0mJB5KBELxYkSZxLImhP1qM3x4dsLDlOY0ISN1qSdrwK21ckhrTEjf1lalDdNJ3lEnhwUAKHkZuStJ0AjrMwT/fwDxSbpEbGhXFiNBpN4+TUuN0zDPh3vgpdRDU8kpDYh17+B/kFpDFuYKcBAMOISbqKQ4OmPkI6nXkcLRpOGs0AqGU0FpTgouNGHnmFonyUNqxmcwrYm73qYr4DMaS9blEQY3V2k5IE+LxXeRmROdJGKiH5YPCCKK0GJd+ZreYwtnMUXkGGs43rZ+F/C8+0mDtxzFgOSGo8QWVaOMvX3z9+Cpo5VWtzSdInoIx+uIpWCQVuXDvBirSYGgImomU1RIozJyEXtDe0jGg+T0jK+PDnBkM1jk8MK8MwBHiojjXlqQyzSNkYJEiIQOT0xLAriIicdQISXqZXRtvolkjbHZl4RoyJ2E2TnCIr8qHR0JO2m1MdHy+c9KoDcgeZ5cSaJS6Kcr42bM6P0NKxUqGzoojzM+scmbKBBUauaRxw8kYVknFg2BxRbB1W2GVdYfZ9C7P1hc3qCuvet7CuWhjX/7O4ZVy2jNm5vtPehGA6uY9iN/HDq9cEqDd4ee/JDR3z/87VPvCXLxzQ7dj58MIP2d/4zUl87ywztcypJ6kD89kbtE88XkQDMFJnVWgwavZXfvKdH8LE0KBZ//gHUyGYqhrrpnGICMaxYZN2v8DLK7jONEvR5hyA0krAMkNiDymxhzkM/qEAQXTZKJd6wLF/j1NoFP61jD401gygZXSbzVy3N4IW531fkqNxIo0BmM/kqWQo4keNYfKvrKwKz8jqOFMh81pLGdkoSls5g1o460bW7pyh5rSipsc5RM7lv+kv1u2gtbDPGCd7SsuT9bKRq4AsA+iGhXqD7uRgWdXyps/LAFFlJR2yjiyL5VARHJs8QLAw1gKursvkXQ97x1RQnQxLUZ2cGPQY25JVIB9esasvCY6yqO0w79Z11kNOM6PFe1ZGRDubfHRrFUqfSbL41KmqSgjl6pEwWvgh2BhxrsN+2BBUQFdtZejjIqAsCKNdawuVrKMgutRsElNK28kMaxuQC3+nZhPIFVNsaRZIozdoSL9JY7C42Vwv2275JJJNXdIU8za+nDUe3MYb48HtJf5vtmm+1U43aN/HTgJm1eX9ShQg0B0Jwau5AYhO1+4OoX/GMER3RvagC78v8bc16uLvWTFIFWinht0dF7LnlTGpA2RqVToDoU6MKQlUqSj458ErJwWHKQQbCNoD/r1pgZtIk/KJEhq5gWrqU4MP/jrOQLiRBWkHB03FQoujDwDPAc/880xBcnLvGLl3QA5gc1LvRFJ88ok+nL07F1OhkHTdQf5fEzdtoHXxDniHv/yWYbcEm6/QyE1JtRijnn/pp9R4fhX7CyfGpjqjsOZf5vSP2YKf9nw0mvXpz/l8NLcI/dl1Havr0p/esDvqjunPSX84mnn059gdDPszs8UJkt5o5HZpzsydeV320x7NiDs3AYaKSeXrDYEUT+ZsNMd/KLZD3BEZc85msxHnYezNxw5PnYwnQ/rTHcwsb8B+9ifupJ9zNh/Nhh5jx5t5szFjf0I8hwxyzvZy7lwSBG/AiwKWRlOWoTgt4JnM/cuSz+LBiPNTSJ4AfjbCUd2DRqZuS8vgLoyfwMjme4Uvwwi28gE9UwhunFFoQQu5gtAuDFya0fvAvWoMBl9j0KBpTvc0igRFQUccUW7YD/y3Oa2nObIUmqWOxemiEZzTZh9nuZ6eWS2j+Pe8JWXYNNUuZ3wSjPNmyY+Thd7BCAVMA1BlN3c4TbXLu9zsl6yXEqkMSppt/OSl85I5j02xo5ckLg58d20+DTmcFLqDAZUH/N0UKN+rEUHEtiBi+1ycfyubJW8Yu9SU9VhVZZ03FRO8cM2hDZ9BO4AjcyMJmrUgNhXIRRNYAH+8ZZTarixS3TT9qZqrdmb++JZjIrbv3jz3y9KWVd9y7Xs1nfWnaLpCorpGqOo+9f2xvqvaWzqd2kR84vLQ8eVzGB/yprK9k8GdYhRMsXvu20/v2tx3bfB7N/m9G722g23LtIX+Z98JszrzXLQ6i96bN+exYd25Oa0/X3NadXK37t2cdyaraU6x6QqzJWvgpty2uY9/d9tDW2CJhJSHf2QV0GhTxqmgUlLTbe6tL4qXtc2QNMylA85CyqPDZin89lEWpdyrsuwzOtaei2YnBwF+VsAGTK3E+2J9fnrrU50IK+ImOhnlYZsWjdu0hMDN3ZtgZ+WkvvcsqtRLdGJfOXGaPL55iqC5Y07lIjpcKL/ReTkFmq3bMnr6HBDsYCvEkEP08e8eKNBWyAGHpBj9+2D08O/Bx2HC30PUl8L7FGPMfniNfjwIdO5AK4mdlVnBXxu28ZUSjyyspxw7jVdkixaG5NJJ/WuSLyEeFyucGczCuQRze+VJSxCqzSbEa3DVN8fp4HhMh+JmJ1kGftowTcXUY0jZYuVxSbF4js5oQNCLJcJCmSremUT4XBzjUcrLmO6QuIjJkjhpchHNcbV6FQTSPK4J/Elkp8bBga/OebyHJylw0zIS3yOcB84lY1kICJZkCIAv6TYdLh2AbcrAueTo0F2ujMpTeVjRBNeYyBS2jN1EVTFf15XLg413KZWCKsVq7QM6Fpasg2IG4Y1TzCKl1hJ/V1oIijGU9admvdVSnkVLplLV5CeM62dK9c70tl/h5GkzNAbheUV1NQ0s1l878IhmkRjsYsBfLI1PaWlo94rsLIbdxVuxlIJmBFuKexR6z0LPd0nSUCPZPkvHHwnoCUVVNiBlA/oZHxLoIJytJQGSASOBDjvTLMg5M9GOMc+pJQOzDomdwBSVjZbRWa6SqwyBMmpqY09lkqrFnrGeb5rh1TzjqM7aT8zzc3niy4BPDA6VvPeXF3QcMpWlHoHdtw9uNeCbIzmZhB4kvtV747zg47uWy/B0VLfhGG1dWbgzCder6vnepT1Y46mLZqUFkTJ7FPEONTIbCyd5TzwjWqVN815sXgRR9H61LHGbrd8Y33xjfMVx/cswitFBpKNlTetU81WuDlPVZDVLUrDRaN/NVZDxdkHXqs1zxV3MOJVRqzzHEoer8H0YfQgl9iqsBgFPLKxqXtpF9tgFdaJH607fbTuIou+zp9u7Tk7zPj2grFd3ajZWpML6rq22tc3qe0hFe22qHBGxuOO/npqKUxEFpEPiOIob5i+MF3HsN/k8ot3ZxXcBsALeRX6Y+R7SBtNH0MhvlsQtrdJegPHn3PwSpn7wd7bhu+ERjP3R7cktw6FgmeyE+YttD/fTm59mCYmv6d4dvg2WxAmhaFlWowEmbuyTJN/HnDUXTz+zzju+gPgaSwf9s8qTD9+J/cKJ3+Ou+xNDYLfz+4rEN2/AnHXTKH4UBA1T3bktSp7VrSGuSmSeEAmwvyiFyRoEIJ2YLKJr0mjqNLkkoI7nJ1CHEC0PtSlbxu0m3ykMtUjSRyG4Dcjfd7GzIMIWcD3tiP0QGy+zY/TbumcrP/Ae8U3m3/mXq1hpeZeGSrI6b9MTmb+LXanLLIp6eUf+MstpHrEINkZ58Evc6kq31D92EjLsF0BCYgn2KYsTSaA0TYSkk9YLmJdUykqGiIPHDzBi4NE96BxeSBRh11nfFUCLNBHyRgN5o4VMApgAPA24kiHiyCG0AsVVtoEIJq6/hmHwFYlxD0iBICUrklyR76iBjYsPzwTrV5JqFdB0rxiksBPT9ir2WolNClPlnJ0xEKZJCsBGFPXEg6Al5S0UCmE/3EqWHUmoJJphplfomOCo+i2bHFZhslouozgFG8ij036zvFed6t0FWkpyoeyQiKKVTVFoueSiAD1cvg8eE9xVHMNYLScmZEk9GEt0YZQ4Ur5Rt1DZThZ5uFGTmsoGMyyATZq8/CLwn/ODrB5k3/m2fsq/lLhR6ynXOI1SJ3gSBYlS7yj4FU9J0Xra50UGqw5IteTDKQIo1Vu7iS4BAMApgIW4GQ9i4nZcyOmA6QQGFf1J7asMrOCIiaywS8S6Fb8fAskDKmIoC7f5h9IZirzmjFqOJylcLrnX0QdFcqC5PxD/8iotie5mV9Hd3EV0Nx8hupt60fHKFb+3iK6ouiA7RGxWqCKfclwMqr3BEfgnatpiobebavkog/UOQlIwBEmVCz+jcgrB+DgXDgDtqVy7Drhtibhu1/Aid7WAjtdxY+Kk5NuA4FfDZKBmfoaKfnboKUTc+F2o5oHRxbMR2dZDCfyKSjaHp+2hhxeE+oTi4iIpWacSr5wqX0KF3IbZ9SiLmkGCb7emB1N+5LuypSBxEbAH4xOnN2xC2TQSlkCymfN1sftdjDfXrVtpN+dpUTjPsu3KN/lDgSX4hXAWQNjZX9oazrf2q9vCS4vLbEMMPf7xaJboSswz5V0JOaKzrkGkmSWG9Q11wuRbXsgoNQPfzU73G6NAcL897txWhaNfvOcGJN39/iIjWXO2qfij47uFpbYKEbYKobQMq2MPmtNdqyPxhIkHeHrvsDiUtzUSzhfL8ISQRnX8UNOEd9EzdvJoIZ7S2VYpqzPYoUVqWriN1aEtjJyyry0RmD3N0Q4w3p+pxyeFmC4dgMGjVEZg6bBFTuLgRPQQxIH4Yf1APVU55C5EMZ8I5ghqIBubLGVDxU0Bf1MPz4exYuo38Ex1QGB0jvHE0q2weMKtAgabp1aCSz8FBwHwqeF8lsvrvKx0WZTxRLKG+QBg/JUu3hhHxldfFdkV9DTb2MX4gTK33GF7+15p0U7SUUkH9T+ZMeVdO6FLnkSrMBV1775GFbeIMt0C++ZAaNzcwMHUws6hYJlBJMNWarVksMnqW4wIIh+nJ4KhhmvFpUVY5VOWTbkacjUV2cm8t7fyDl1T4k6lhTIDKo0KoTXr8StWnZXPGdhb76vXjTfKcKvozkkuDu1mBKW0UlnivGHslbdsSOPAulpt7+NEUf+nRmVxiDo4KXyaSn2t1NZ1vbaiONeyrq7rdFX6WG/R03Wdllbq6Fqro+tqHUMhoYbqpdSsQ9aq587qIpmva1Up13VKuVdZQtVkLf+lGcI7gR+SX7lTYk9rAJM0jt6TipX8KspPnCUCJ7+vnJhshf6PiJpa5gJXd83POsXu1U5svLK6DSdsFT0/J9ZGk6ylGf22weS5D0UdtM+Nv/4VzVTcmFCDIYyrVShVU6puHrUr51H7yzz6ZR6tmUdPP908urfb5GlXTJ72l8nzTz15nn6CyZP+XwyGFdepkFSKUzRC8iH7LW9WEjJwQtcFOZp8HTtjoSrSVlC6Y5QM95tsj4yJkbvSGq0aEcwlwfYlvMJlLbzs4KbqjAP1k4XriGjwgQZgsokMr/iwp5rj9SLWTQWWLBDc5Ix31yA+/4mpp2JHLmL8xSVK5wC5Jw+3p9IUJUS3BSzNVThiLLPeCstOzSWrIBVi3p8sspItDjMiRfwfOuhO4RSD8yaHxCm96R3DNxLSRo6ioLZjPr9sqnSRgi/Hbwrp4zh/ypyldrtc8brFpRyGMmtsH+Z5UC/yVsEqkeHp1U85Dv0S8dT6FZ9fZ/SmpbOuUewpd3iJiIec79Ie/eISMiRwoFnzyToK5vP+wZBOi8UiebjUqLU4/n5+FYGeJ4oQPrUqcnN/FbnZSUW22quqjogItUpSruH9lERC/PMoSXbJmhofbRnaMGeLs1MELc9rFyiL+zAxni9cElRswMLQZeg9AYfTq1wG9PxrsylfpOiHlKqwLqeuqLCC706fUpbQ2IpftsBXkKa3d+arkaZwK6o51YLmK5E7wOK9wN/Ra4GpN59dDFwBncZOmOBO859i/5IFANJoCWPAvIo+TFyv4mhJ4vSmYfoL55K0Y4La74eXeL8L3XMDQvLM5g4E2u3s9tH2H1G0QAL2joh41Vib3oKcgGVCMfvLtZmJmzWHLOq3rhO4jWsnbijlotX84FZcJt4s19mRQJFS3hK7kWLgFbSyu2npKQKUGZgHZqYsDH/nRhLBJTnRS5JRODpWzfLisyp0fsHvcygrr/WD29LAv6G3N3YGZIHjLNQ3q3AFvZ+jpUBuvRO50i6GIHjuzEggbu7gq0o0/ZVy7oHPLbj+voB+QWFoSI0unQf4Ka2f05RMg/DmqzfYkVD6SyGeJkKxa5If06u2Ec7q2FAD4cJkLRbqMLv3SBiKTCoavLPNpsuyDaszxMG13IhN1DiWIXeJZnYWZfP59wzR+ihHqutHTZ0kxB6R3desFVkgqmN1J+Rei9D/1PbiLcUI2Zb1Nejbg1sf9Y+qnx6PjydCbbdxopquRYRSwxxGQZ+wQT5zImiGjhWEfYTXr6PMhBHB0Co61/Lcoy1NWXJ+3sPY7iIxc/PZ91L9N6uUNGrGUQrkcdhsTyyPXJpa2ppxmWtUjMO+tphYmj921jidpraN+2v+nTW4bFjXqrBGOWWZ7KCde7IFDar6/N4qUaBv7V0C6A4qVIKeFaN/NVCmBXUwMPZna0Mmtnc1pKS6uVK2I6qVqMSSViqqLPb+gvJ9hwgNBWEyZjC3FOhIqEcLj1lhEqW9aVbtAKSBl8fRmiTVFvxH+QhFAR03cJLkuZ+kHbBYwNIN5xGKkr/BkFtOdwkPcTB2xWHtNlHuBbFdxW84U4riKysJAuu7VBvfJDGFK5yVsjiNxtv/Ch/cFn1kc/ZW2cCDr1+UWKsqlD6bIfRKhsyjqIbJXtIwldxsK05ZShyA7RGrznbWsrhtBSBJyVJc29CLgomToWyTmnlumIqcmN7cU04MOZcTfRTFVDKrxcQBqsSUZVeKiQOoYsqTyx4W2qt9y73Cq1USPkEUM0SdfBnRsuxcDJxCzhMYb/CVEvpWBblOJYO9JAsA6EBVL0nKkorQh6Jb1YB78kp19bZlfr+ESknq3kJcqCL+rzY5jD3f4q0NOBARGC4z3WjphKL2q/sglwddjOIEQOETDrmcYjH+ZCkdupHOzO2PBTsOZ5YOruM93zDU4il+vNGmM6AH6m28gUZcIZBr8NOST+MS/3nZu7AeURLiwCkTzzUqvzRcPC+D5lR5INhxGCiQtQOBkM16umlrcmgnN7uaLN69TSe8KWdmtdIcqBJJkPRRmsb+DCbzhklbuCU2rRKIm0f4zNanC/BxitqZWwGpd+ozqGw0/x8RvpzVVonw7EIA+nzzgPrz9FjyPGqW4xj5G02VHaxshO0kkIwwk8hLGKiwnlmqOVV5UCbyT8cDDkZlHvAyLAXobuG6svuhENo5WlemxC2w/AWnTxiHzmhWmZg5gByeE+5YX7+gj7rBxKGMdBgAwo1TNNaP7QXz0GN8z8sPL58EPjDzWjoTLCz5hOAWvc7E9eA2o8RdlXZOmsZeQFBvtStCWQxAsPkrdoLkJm4WphNw5CUeDtKhPoLiQOVca+DFWKveRUJpi8e0sRbUzuUyVEhtESZbwsnxM+07LUTHUpSbt7axq9j5MvcsUNGgQYqmQbMw+N5oW50B2l92p0sWyg0RH1NDQ9YTHr8tKUshBK4teHDKvRIv2dPsRbk3aUsOnKi7SiQ9n6+CrPIZ7TpNLvzbbXrMieYIuJ1nqzwFbuRTU/P0qHi2I4OiVW5lhdEv6TWONFrqsCC5QIIPCYfK9ag4PZtj0YwCj35KmNmTkmVUllPgsm8BeaN5nKVu2UHQAxr6bRuSSMThqG7BoRTuYq9h0ghcXgAurQj04VOKb1XQF4ZOWYLi0Bnz1aBtxPIVBJEaD+II5FhKQW+jnErHiQmooflCtwwlxb4WprWrJUwzBPO/i6PFG26Z6o8RSrsD8KoT79WaXZ7B18NpauODH3rRh45HrsHDoKVSIOlm/goYfOpK2tyjFmMbh2JR+LmtOPnxn2UO8Mh7t0rSBYsBKgVVUq1+bsdNklfrn6nbR18oiRN2nr+h2tCSHEQspe4yQVutDmJL6A+VSjSNr4X3NE5K15t83JKu/v7LO9F0VmlkVjQTXbNEPRRrKG5Ce6htyapOtWXBGfoYLVC4tUq0sKQlNMbfByd1rwS9VXsM35RCPF++ZZir1QLRX2Bu420jJjSgTJ8ofXBboXgbb4lDkBIWCbhbXyrfYGXzO2OUEIB7RZ3GVo4uzEa68UCcrZSKC3GL7AcruBx4qCp1U3Iiq4MWciVrBrf89rjAY+vMAnl2lRMkZ6D1dd6TWi5x4ygIHks22S2ddHUl4FuGlIOWMSNXzrWPr7iaeL+SE6bmRi1DdKRpOc/CNPq7Tz40bjXoQDOI3PeQEhIHlKgguJEPvZfluYhWCdUMlKkaQ8uuPsJ9piA5ccMphrZY5f6zZRQfv0ljWoapOQGdu6OsW/KXemngNwqJcH2k4sVVAVbdBJ2fhry4Vu5IzDcf5TXbpVAq6Gr22Fq0ZKVvcTfppll6ILpiDa1URqpYL9uLoAc+theBzSEviOmvnZTKrhZwzZU5zekOiqCKWghOgGkEEEM1Dy0m4mULZTllppos+SAnoEf+IYsSKNg8XYdOLxQrugP0dJGPU8MyvvlGEpkIfKAAM9dL4Fh2EBVp8a0PpnTZrQLDvWPFc9xt6ViuVlZ1RXc0zpuWTS0HOxV9UFV0VWUliWwqWuq3vKV+yJzzmqb6LW+qHFpsqx803rzCG+239W2VrRd/ZGP99jGNVR5e7tBWv92hrYrFcbPyCMhO01fELAJ19tpxitlpgtnKCI3TbZtDX2dHDe45k2b45em0fCoGAe4/OV4UsyIrdKpep3pxXb7+tZ4L/c0eeQBUnnnMJ5hBvCMMlivauOUyZLUypetOdH92m6wM+bks/ZEi+UTVNplUHMQqoamPAuhvCNlJVz9cERLodDUbH50ghXLLgS2PBKnzG7oYDLfZYSkFV4jN75Z/SuYOKI4aEi0eoi4Z7U18eKHIZ7SVR29qLX1RJmVA3NeusMtdA5g1EO458B3s6LLn99OJiBKhqXoQuszRqdHfVrv2idFX9U8q9biCXbzNpnSdjcyu9Hkgn2DUHfWrrMex0d5akYNtFTmtqogf3qki7e0V0SxmiiR294HLP+639FJ2Km/VSC7tmC7F/U2M+2EI1zhUdPihMuY1FAlJ0FIclwWbhcL+U13u+ZSl6XzZor00t9UunSTxr8kRe71lI62J6XZR1DWgNoBRurCWBap+CslT/mbPp7uuVr5dSLxoVLP5lQUvOFCWooHLw8UiKEucytczla/Q2/0SvdI1evIleOVr80r5jIVU3S1WfTFe+V4k+poEe7SBT/541wRoCW6JOBJeWQLN4bOucjI2P1h0j12klUd4bNdZmlVQxemdWrAFjTiyNjetzmhAFpWwgsXqh3h7R1v2nEvrCvqjJnpgNI7nAT0MbLLrn6th608baSRWeY5ChcRa5Z652e0MzJ3iu9vC11V1XqVYIG2k5Vo5srFXdzYsP+QldYJa/RBBqpRDhKnYgLynLDr/jNNC6RJzzPk2KPniO2p8Va0zsqJxxJPUunfFvcslMO3hFwOD8tgFwNXMlzNtus8LZqKsstlNAxuyeFtJv3bzdgk6C660B3U8F16sbdfBSdvfXRKKF+uUi5b03dYByu6SOiFsVH1gM8CfRyVYff9tWsFjPf92lcgmflkrqrYs5W1ZvsOfn/G95W8FKDbRUTmJ2XGa6/SPdIkIvZnubZrUCkuvfGoCvI6i9GXkkUazcxUlKXib8zDpZAGo7EZG+Dk9PgTL2V+mp8eHaUxI4sIU0I5XYfuKxOT0GDe4G3SH1cn+PAo8fMrjIgTK+6fHVE6nx3RhyUDL4WTfvSLue6jDvhbnIo0uLwNEPaRIMnl628eFM5uBF7xfl/VNkE7fOesOFYTBr4Vv9FvoE/7rn/+zbXWssfF//4/VGRpn//rn/4bvCWT+65//y+7Y50YYhX+QODqyh1B1KOM0/4vXRywYHP7wgqwhxSOewhSe6AaV8S5cVAFoxtp8yMR9mjwvE/9FrjgKsvIKxv7p62wuZk3X6XQyvrXNRtuUNwooauC7VGkPIzclaTsBHGexf5q/bMUUn2oP+0LlmebBizm90L9WuaYMrAM8vImiMHv/gpNIlzw8gOOi9OhHw0zJYomGBdIhcQwCAxMOj4YUxiZ7v+A/3vz0skNd3QYS7PAT3Co9VnezKfVqoIYo+fsfhtwjOrrXKxi7xdMX2Ku3dJesLlkzSlXar2mXd0kE+nO7j7sX94+M/acsQJZfgQP2PYb1wKM1HLpU3+wYP6CJecjScdMVe7eAhgT3W8a+8FoBUvz+x8nL65F9E8d9uxs+Ofzt5bfvX/0Yk6ufh78cPBsfDn7sP3/VvXlzPRrEi/6Lw+WjD6vr/nL5/fPRu791308W6bV/HYyd70cH4d9e9g8eLZ+/ur48OcmLog8qYEm8R2KG8ngB5j7627ePy/8xOsLtCQB6u+/Y8JeNBVjwow/eyH5xehZSzm738eAAkgVQyKVdmH4ClGEDDqbSUwOQCu7OPnQjSuocft9U0bLKtKwttJTjSkgQk+UrjGilMjEVlx0hXZiIIbENQ1S3Pxr1Bla315+MB70B5jlrlIMuiwdSIZt5cPv5tdPIY8eyLWswGY77A7s/mUx6Ew5Bb6Ouopk9y4x1OJsM2YGAHn0v1u712Wffpp/DEeQM8T/6OYafI3z5ln1OxgBs4RN97FldC7+RB9tmr+jamDDCdyAHLKELCV2rBwkT9l5tH48mYAndHoOwung4gf6PFTJE9Ak+G2hZNMHGIwwji6ZiwmiALGNJY4rRw5/dXv6NxHCC4OBY4hArPaQv4vaAWh8p2vb5OSqgtHkfRDRial4VA2ealbU5f/RvP8Oh6oWdZx9jNLVgZzIIvU+E9j1QuUJpi2e8UDOzhExRt5UBv7cUYn98Iefb6/pfoQHjvQzHMjfnG3EI5sMt/ITJ9VSd0NlfO9oYylT/D+Mp3T50ZDx59YthcWLfXKZTfSmMAZziqdUJQz8LLhyhwUH2Dd9DYyi+yGeEMZnZtjPrj5yx1e+Pe443sfoDazCezPrEHs45e5/JBDMoGPHuaYq50QKdmos6+4jmAXjgLBPMRLNN1yYKjb8YHOiTmHi1DFU35mc2BWm1GmdnwPfY6vUH3QmMIvDR61sje2L36Mekb427k6HRMoSBvHhi9QxSu0O7P+z3YQArRnIGP+gPhr1Rj5Md9e1h3zBEZLtjdcf22EJ4ABmOJmPQQ/YBo+W4b9m85MFw0p8MhMddke1+v2eN7eGYIYPWjmycFgC+D5UYjwxGqW8N+rY1xrfSuQNFlZQ3aLNC+h/XsQa9ruNZ7sgbeXZ/NBjM7NHEHRH8Hg67Q+/zdqzP79v0bKr5g3Gm+uPBF+/mi3fz5/ZuXv6cHFyP3/3w6/PDyfrHRwfjH3/8+Tq+SQ/j/uT3P348HKWPnl27T9dXfxw+fWH/fjhY239/FYJHFA/D0c/9w9c3109eJb/+8PLd6PDdVXKYRt9fv/ri3Xxy78Yadq3hAKaqAdjp9ngseDearG3eTQ/mmm5vBLPPcGL1BuicKO6NhugX9+aLe/PFvfnc7k23O3RmExvUuNvrD+f2xLFHs9l4NOna4OiMvC/uzaex8v6/d3BG4ClMrAnzB8Z9dDT6zLPo2X3L6vKc7njcG3QHhuJmTMB1mXT7zM2YgPYMuI/Cx/4xdX34KK8gjwcwqnZ7rDAYnUEZx2PuEIHrMjBYzqg/GNu9sSG7Vj0oqzfs9yiIPRp3c0pDazQcWCNeh353NB4C2x/r4Hy0ncuuS3nm0bsaC9Ns7no2yGoEbmKvP3aHznzYnY9G3R4Z2E6/P6SGyc4xCRF4q58lAm8dDs6n3ByP3BgscK1BntnMKCsOxMEVK3m/k4vzArP3S6hPgojuU8i+Oy4m0GLosyvTIicmMNC55Fc/vWpI6DnReexc8kOQyuLvU/75HYfAOmTQ0gJiRhjzNeen88ZVzk+zB12zA445Zfjky86Pb54B8QwbXzaYsqeAo1Xskqf05pAKGf4Fx/N948BQ0Pm9PKJYcmqduR8nWdm0ZoBQ5BYOykZuCa2Qt/kj/w+O9cyg</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_5ecb69e45849446495cac4e3367e5c4f\")) .filter((elt) => !elt.dataset['step1']) )[0]; root.dataset['step1'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>\n",
    "\n",
    "\n",
    "## Restore checkpoints\n",
    "\n",
    "Note that you saved the checkpoint as a Flax class of [`nnx.State`](https://flax.readthedocs.io/en/latest/api_reference/flax.nnx/state.html#flax.nnx.State), which is also nested with [`nnx.Variable`](https://flax.readthedocs.io/en/latest/api_reference/flax.nnx/variables.html#flax.nnx.Variable) and [`nnx.Param`](https://flax.readthedocs.io/en/latest/api_reference/flax.nnx/variables.html#flax.nnx.Param) classes.\n",
    "\n",
    "At checkpoint restoration time, you need to have these classes ready in your runtime, and instruct the checkpointing library (Orbax) to restore your pytree back to that structure. This can be achieved as follows:\n",
    "- First, create an abstract Flax NNX model (without allocating any memory for arrays), and show its abstract variable state to the checkpointing library.\n",
    "- Once you have the state, use [`nnx.merge`](https://flax.readthedocs.io/en/latest/api_reference/flax.nnx/graph.html#flax.nnx.merge) to obtain your Flax NNX model, and use it as usual."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The abstract NNX state (all leaves are abstract arrays):\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<script> (()=>{ if (customElements.get('treescope-container') === undefined) { class TreescopeContainer extends HTMLElement { constructor() { super(); this.attachShadow({mode: \"open\"}); this.defns = {}; this.state = {}; } } customElements.define(\"treescope-container\", TreescopeContainer); } if (customElements.get('treescope-run-here') === undefined) { class RunHere extends HTMLElement { constructor() { super() } connectedCallback() { const run = child => { const fn = new Function(child.textContent); child.textContent = \"\"; fn.call(this); this.remove(); }; const child = this.querySelector(\"script\"); if (child) { run(child); } else { new MutationObserver(()=>{ run(this.querySelector(\"script\")); }).observe(this, {childList: true}); } } } customElements.define(\"treescope-run-here\", RunHere); } })(); </script> <treescope-container class=\"treescope_out_0a649299f88b4449b5c992f7be3625cf\" style=\"display:block\"></treescope-container> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_0a649299f88b4449b5c992f7be3625cf\")) .filter((elt) => !elt.dataset.setup) )[0]; root.dataset.setup = 1; const msg = document.createElement(\"span\"); msg.style = \"color: #cccccc; font-family: monospace;\"; msg.textContent = \"(Loading...)\"; root.state.loadingMsg = msg; root.shadowRoot.appendChild(msg); root.state.chain = new Promise((resolve, reject) => { const observer = new IntersectionObserver((entries) => { for (const entry of entries) { if (entry.isIntersecting) { resolve(); observer.disconnect(); return; } } }, {rootMargin: \"1000px\"}); window.setTimeout(() => { observer.observe(root); }, 0); }); root.state.deferring = false; const _insertNode = (node) => { for (let oldScript of node.querySelectorAll(\"script\")) { let newScript = document.createElement(\"script\"); newScript.type = oldScript.type; newScript.textContent = oldScript.textContent; oldScript.parentNode.replaceChild(newScript, oldScript); } if (root.state.loadingMsg) { root.state.loadingMsg.remove(); root.state.loadingMsg = null; } root.shadowRoot.appendChild(node); }; root.defns.insertContent = ((contentNode, compressed) => { if (compressed) { root.state.deferring = true; } if (root.state.deferring) { root.state.chain = (async () => { await root.state.chain; if (compressed) { const encoded = contentNode.textContent; const blob = new Blob([ Uint8Array.from(atob(encoded), (m) => m.codePointAt(0)) ]); const reader = blob.stream().pipeThrough( new DecompressionStream(\"deflate\") ).pipeThrough( new TextDecoderStream(\"utf-8\") ).getReader(); const parts = []; while (true) { const step = await reader.read(); if (step.done) { break; } parts.push(step.value); } const tpl = document.createElement('template'); tpl.innerHTML = parts.join(\"\"); _insertNode(tpl.content); } else { _insertNode(contentNode.content); } })(); } else { _insertNode(contentNode.content); } }); </script></treescope-run-here><div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtGotW2sr2V6Z03QNcJQIKKirrBuRlq1ax1drTxR2SSRhJJnEyAfEs//3uScIbqVZPreeKawlM9vs1e2/d9cTAIkVFcEI8zXFJizuOQH8h1/GooA4rIE4sLGiP7CDDYSJlYJtagwKyHeZ4LtbgvN+hgqSCLwXkcjixqCdSAemUGLhwyhwGx22sdU3u+ExPaY7l8EKIuoOib20LAIAe1UWngAwqAIwJwsQOsilLReeZdPpfQMu5SXn0ljIT8ByuE56Cox3kYl2Hw5RFDFFAWa0jpWEk1SHU7MBJRslJfkxgCsqN6EcfUj3q0Ta1qAAVsS+cEWyKMsEp86gm2ZLwaaTX3e5aaMfdkR1T3GfAk8OZp3HqCiQNsRfHrmtRDUvTrjmaINJMnGA7XkwkkntFsDzw8wTSicE8tIdEh3qKScQpuOXI0UkiqXQcTyjBc1CNCNRyCZMqq5qkKpG+fV/0pI6ZbhF4zHzL2gk5KCBm03EYnCb6Du8m0aQMzjkcyUdTx4Jq8tAl3HC4jZlGFOb0E8kgEIBBYu4JSoVIu2g9mwQ61ECJGakVizBTdNDeHkpLkKWicyJ8zsDuiFgeGQvW8ZmUbJa016GGkPIFAPLDHfzcwyEB4cd0p69wcu0TT6iM2oG7qhzbJBHaJClp7Mwxcn2vE5pxZ4GOQxZ7oRpLtHy4DFKK0JHCMU0rTN9WkGIQra6kJU+IJVYR6UGAR56U0gXflS4ZSKPHeEwKFAErmoU97yNkcUQ3ERvRbNkQhrEh87sk2BPCP4jx4u7aogTQaQ8FBPdi03UmhgRug6bkZi+WjkHqcjEP4jAQEYzB4NGyZFhsgYTEGeoeg2QM611QcFq43eakF8RPUH/e57eyOJ0GrSIAzbFtQJyAwMFLKj8DggvMEYlCx+kRnlwAH4Ez324TPkEwvb2VyeXHAJ6sMuYkx/VMLpOTAIZj6bgNKjLQ/Z4aDXJNgbVCqwC0Tj3XwoNhLZ4FREVk4TaxCoU2gewlExJowWtnIb+wzKYyss5G9RnsN+JFWVB825YjC/u9PAOrzXPWMe96BJsQEWweOzD4zFEHe4liQLO40A6hk7QO0bpETybRv5NjGSTqYqQh/JSEwX1RQPE/s7m2Fn9J8aaR7hUy/wuElH6UjH3uSQe6DtyahC/gS73nYxukQsAoFeS5d1+MPw/XsXqC3Ih5Lgr1Wgblnmg5rCXDf0FqLUslJZuT2bTQVejJ4ocenxVRamVjbkKfE4oRJPTdE7lBSXMHbV8IaDAWFaDx40VBG0OxGSgwJHSZi4H/JJkNPTbTpMYPMUQFxRZqDuy2Y3no2BdSXx2VQ0x4dweQGKk+aXeh4QzQPRvukE7QWmImAJ1ij+ijNvU9ScufnfkwD7GD9jCtbBN7VsswPxZosbjcjTGVPvZaGnSOYNgRPjbE1FUyrNPLeM7gTLOcND3qYZ5IpXQscAozcGzQgCQnjyUT2VVxzIbRHJBFGQ8RsBi0zCnHF49TZSQBOIYS/d20JAFL9I7arsMFZnO029zpEtaSJ+Ni9GPrTqBN2HPo5jtF9iggmN7SoMPVOWGRqNMzBNCcBnym6jqROtFFOpWqGra0BAw60GVn3JugMVM8gSX+SN6/TZJo9Aol0R0BukspJo137WOLQd/agtHQoDdAZCpNtoI0gSYdcxC4jzmDxGsNC/vQF4aBtcz6AkAXuty/RlMfj4Y8Wb0iI0VHqbQSlNXxDFoIBkPMUybHOgW3JVBmPacTcxU5ENImQWkQL691VsMQh/ZUFozgCEVmnpNlrrI+T9lGcwV6qM+d0u9A6MpWdlGNlVceFOKFMGFzNgEFXO8jFFIAb1jYhVr4437y8ZfF/RzGgoZA5AaKhn4vzHPIsYjFlCnGM9YiU0wPMcr08ISWUJhVlU2Nc4sBH8LsXjrPuMGQQyV6p3KOB4rBHRsGVc2Xw5EiE95TetiCaTaRTCqeA2NsUAbkOCrflfDKlqPoAy/tWBySIjka/r0OIUJuCEgflZvNptSmKc/kvB88hGkaVNZIc8C0xH//EzUKGhkWpMc3DWEdk5ykHbmNreisHy2bNuQw6XGtgHxuJeQNVpDP1/qOYWR32nBH5jdW9fR27dBUS2rwapyoqhN8Kp324Xe9qqoVddmrZKuq2XU+6I1Kqdz/qqpnX8sH6mGjVFar5k2j/rEjvNIhJeZ6df8i+7GR/9pruj79dJg7yxxcNE6/HPbOD2/Fp0G1Wl45N7tntLSf7tD9E/+goteu0vX2mtFr6O71h3zn+pzSE/+Q1Tp147NQP+dLR3xDrTZYt5LXPvs+WznNXWtet98zqtba9Y1ZcbbM9kG/tpWpq2tMPc195Pwgc7pi3qZP9bR6YGTMo81yv3aVNdPOwD/d3LQrmXy/frF9bJouOesONkijfZvT2vy4JrBqnjSO+vvYG3gnfqNxcV6p9tVPJ27jq/55bW3F3DzbvFgXaePDp2u1lwOaH9WjTfWwr9rm7Wlzxb9sksrFTdbIa7dHG6f1Qc4vqR9uS1du1V2n9ZNyJX3pf9pobjKj9LFSrx7aKl3Z6lWyHZbpbK60v/Qvrvp13tuvfS6zK6NSMcXKsXZpWZu57fJBv7TV2d44PKw112uXqmk3clelk21xViP17Uqp1Kit75sbp2tftUFbrYFPv3xYU09qWCWHZUut31aOzUth5kufzOPjxn6pS09ypFq6KJeqGk27He64DGLDvazsZ24z3aZRNkRn8IHVdVz16kb6yK5VjvIlXb3+8sXFwmte2rqO6XbWuN3e+EyvrvOuzfPHztdyk/Ka3TuorTfPm+vVSlYrnRhnK3XLcWsbVa+fw+Z1fotekuaR5Z6zUr1B9ENO/PPrWtnOnFd5t9m8yWXz5+deXwWJkijY9olEPAjruLwa/wu/RtmPdceFvmCcksGOUlGUJRCrYc5+B1rLtz6dYGkWtG5hVwm0ITyYhhJhcze90oQUPHNk+gJY1PzJMw/KgyQhu1nZAuI+pgIx3KMmFg5XgLLbdjDXlT6ngpzB4JcY0wJlI1rjvRk0JInYRKsrN2bA5YzaBHrixHClOofHiQ196xzq3SrKptPpoO2A4gsdSCIY2hbznehnY2Ph5Lg6rGByyRhD71EVUwsKm3CQBH4XVDZoNBj0bFCNKdiMYF224yuTtou2fz/Y+8nWfrj4m977zDZJseJueEPvUub60U0TC+7ktnMTW0gkur7hYXh1gxAB8jTf6Zs2VvzDElJsgFgON/VwpoONFQ0L3yiM3cg+WxCLtpUhzfCtKY8Tf+2uRVpNEIsPm6z49LE8alk9a/o4Njd5xJY9n3k4uVEE1d/fZDd3wp43E36JxC2g+331rK5ZbJHxIBZHDivLoN+LPzLrgz1vMo5GU+Je7Fs80jX+PYaCW3ovNjFGFtAf174jdibAwoMdNDf2QlMQxD3U1Q58jsxcnHb67+TlLuGMWC/j5F+Zfz0MjRpINJ+CnzDHduKF4u1bPHTAgyNvjPB8MRibGmQWBk30x4tYEW6CTB4l8huolHyxqA7mg72/J0ofFp3Lo+4Kgm6mznewS/alHE3BfU28fLyFQ9ajoy5EezX1z5N2/7sj5XdxphJo+5MuDZFf88UW/sU0VtyIpFmdz+DR0ggK2SML4Yv581v6+5NcCvhP9Or9F8dPeGWZvVdfjU8yT/RJ5nl98iNHxUdr3fjS2/2+pmvyvyBixSwiFpEIHozm94OPqZpjquFb8p+Voi9Zch+enHpwuS1ta5g79GcAnAjnAsNysFjPhl+Srz2BA81+1mMB8itO3acm7Ct3/su0u6/T4UuH8zn4VxsYv3rufslgyCAqiP2zIXH3Wi/tX7PUe1KXvGj1mv0/Wr1mH+al7Nvq9W31+kzx9rgrIPu2en1bvT5TvD28F82+rV5/w9Vr9il7gOzb6vV3axGzT1y9Zt9Wr7/QJ5kn+uRt9fpPStGXLLlvq9dHeuzBq9fs2+r1Va9ef5t29231+hsHxq+eu1//6vWVufrXrPReth+Tbv3pZuzuOXN9BKnTXvF/cDuGNw==</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_0a649299f88b4449b5c992f7be3625cf\")) .filter((elt) => !elt.dataset['step0']) )[0]; root.dataset['step0'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NNX State restored: \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/cgarciae/repos/flax/.venv/lib/python3.11/site-packages/orbax/checkpoint/_src/serialization/type_handlers.py:1251: UserWarning: Sharding info not provided when restoring. Populating sharding info from sharding file. Please note restoration time will be slightly increased due to reading from file. Note also that this option is unsafe when restoring on a different topology than the checkpoint was saved with.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<script> (()=>{ if (customElements.get('treescope-container') === undefined) { class TreescopeContainer extends HTMLElement { constructor() { super(); this.attachShadow({mode: \"open\"}); this.defns = {}; this.state = {}; } } customElements.define(\"treescope-container\", TreescopeContainer); } if (customElements.get('treescope-run-here') === undefined) { class RunHere extends HTMLElement { constructor() { super() } connectedCallback() { const run = child => { const fn = new Function(child.textContent); child.textContent = \"\"; fn.call(this); this.remove(); }; const child = this.querySelector(\"script\"); if (child) { run(child); } else { new MutationObserver(()=>{ run(this.querySelector(\"script\")); }).observe(this, {childList: true}); } } } customElements.define(\"treescope-run-here\", RunHere); } })(); </script> <treescope-container class=\"treescope_out_a0dfa5fe30174a9c9b95b75954d2484b\" style=\"display:block\"></treescope-container> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_a0dfa5fe30174a9c9b95b75954d2484b\")) .filter((elt) => !elt.dataset.setup) )[0]; root.dataset.setup = 1; const msg = document.createElement(\"span\"); msg.style = \"color: #cccccc; font-family: monospace;\"; msg.textContent = \"(Loading...)\"; root.state.loadingMsg = msg; root.shadowRoot.appendChild(msg); root.state.chain = new Promise((resolve, reject) => { const observer = new IntersectionObserver((entries) => { for (const entry of entries) { if (entry.isIntersecting) { resolve(); observer.disconnect(); return; } } }, {rootMargin: \"1000px\"}); window.setTimeout(() => { observer.observe(root); }, 0); }); root.state.deferring = false; const _insertNode = (node) => { for (let oldScript of node.querySelectorAll(\"script\")) { let newScript = document.createElement(\"script\"); newScript.type = oldScript.type; newScript.textContent = oldScript.textContent; oldScript.parentNode.replaceChild(newScript, oldScript); } if (root.state.loadingMsg) { root.state.loadingMsg.remove(); root.state.loadingMsg = null; } root.shadowRoot.appendChild(node); }; root.defns.insertContent = ((contentNode, compressed) => { if (compressed) { root.state.deferring = true; } if (root.state.deferring) { root.state.chain = (async () => { await root.state.chain; if (compressed) { const encoded = contentNode.textContent; const blob = new Blob([ Uint8Array.from(atob(encoded), (m) => m.codePointAt(0)) ]); const reader = blob.stream().pipeThrough( new DecompressionStream(\"deflate\") ).pipeThrough( new TextDecoderStream(\"utf-8\") ).getReader(); const parts = []; while (true) { const step = await reader.read(); if (step.done) { break; } parts.push(step.value); } const tpl = document.createElement('template'); tpl.innerHTML = parts.join(\"\"); _insertNode(tpl.content); } else { _insertNode(contentNode.content); } })(); } else { _insertNode(contentNode.content); } }); </script></treescope-run-here><div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtWY1X2sgW/1em6XkLPCXyjaByXkC+bNUqtlp397CTZJKMhElMBhD3+L+/O0n4jta2btvdLZ4jMLlzv++d+xv2fT61SU3mHiG+5rik7zkOR38i1/Eppw6rIo/YmNMx2UOGw3jawENqT6to6DDHd7EG6xOLcpIOvlSR68GKTX2eDlin+dSFVeYwWFaxNjA9Z8T0tObYjlcNt+6h6JtqAwHwozq3qsigHMgYJ4zvoSFl6Wg9m8n8B3g5d2mf3lNmwj7H04mXhqU95GJdh8W0TQxeRTnNEtowkrYINS1YycpFIY9xTMG4Of/oQ3pMfapSm3IwEY+4M6dNU8Y9ynyqCbEkfBrZ9bC/E/pxf+7HtDdiINODNV/zqMuRcMRBAruuTTUsXLvjaJwIN3kEDxO1ZDJ1UAPPgzyfI50YzEcHiFvUl03CzyEsJ45OkinZcnwuB8/BNMJR3yVMmKxogqvY9OvvcU86mOk2gcdsZNt7oQQZ1Ow5DoPV5MTxBim0rINzCUvi0coyp5pYdIlnON4QM43IzJkkU0EigIDkxhOUDjfto3wuBXyogZJrWss2YSa30MEBygiSJ1X3CB95DPyOiO2ThWLWiAnN1ln7FjW40C8gEB8e4O8RCUlIP6Y7E9kjtyPic4XRYRCuloeHJBn6JCV47G0Icke+FbpxL8bGmYiD0IwnrHy+DkKLMJDcMU07LN9+UGKQra7gJVaIzbcRGUOCR5EU2gXf5QGZCqdLniQUiohlzca+/xaqOOKblOY8+0NIQ2km/CEF/oT0D3K8tr8TVwA6HaOA4YG02mckxLEKlpK7AykjQel6fJPEYaAiOIPBo6eKId4DSbFnZrsExRj2u6Dh9LGqemQc5E/Qf16XdnM4kwGrIgLNGQ5h4xIFDl7C+DUSXGUOT1YtZ0y8VAx9RO6LJmIuM8xni9miIAAriOcRve9CGySWY0NPWyYsi7+oCwdmVBHlGLqJ2GwAOVbBfAZ+eaR/g84rZP3QY0CtUx+ETmd9ep0Q1ZCNVWJXqyqByiZLWmnBay9WXtiC01nRg6PeDb6dy6IsaMyq7Yim/6jMwKObknXsDXyCTcgWtrk7CMbakoX9ZC3gWYv1QxhAzSLagOipFPpvaqGD2Bq/aUa/omFwllRR4rdcUdUS31O91U2PKln6BkqKOArBI88XAXQdOFGJFyOX+i8nNiiFQFA66AH+Yzn+MlIX5nFyxzelyNTvG9Tzed9hfZH+MaX1VCnJuaKopthQoa9WP4z4uorCqiH2TJiBQjWCgn74SmnQD92pOuIcho+4BrR4HJe0EpLWqMCRMIHGE/9GsgVdWhtgE8cYsoJiG/WmQ9WxfXQ64sJeHTXCnfDuTqEw0hOiDmAYDTvvEM4XKxg7MeOwnWKf6PMR9jXJiL+9zTQPdwejY0aukOG6lWF9xFgR3+4WO+UJ9vsanAPg2Pl+bPCV02PWp5+SubZnVeSy69EYe8l0WsccpzGDwAbDSWp5WQgRE5eH2SybA7Yo6yMCHoNxOu2M+OeZMtcAAkOJ/mpVk0AkekWHruNxzDZ4q54zIKwvVhbN6NPeXdq25M9ZmB9kMb+AYnpfg+lX9wiLVF3FF8BzlfCFuutS6UQH6UqpatjWkgCCYALPunfB0Cb7HIv9c33/Mk0iWBZqojscbBdaLDvvdoRtBjNtH2CjQe+AyUqZ7AZlAgM8FnPRBHsMCq8/a+yzWBgG1rL5GEIXJuA/54jQiwCg6F6Rk6KldEYO2uoCn1YD0Ii9tOlhnULYkiibL+rE3EYOpLRJUAbUK2nWdpjiMLqKhhEsocjNG7psdNaXadtoo0HP7HmQJxakrhhz43qsOPKgEcfShMPZEhVIfYxRyAGiYWMXeuGn58nPPywel7BQNCQid9A09EdpXkKPOBErrljgrzhXrAIceRVYoSc4rJvKVqBePOFzhD3K5wVvNwTgRK8Uz8NT2fCcIYBYbSSAkywK3pfH2Aakm0ylZN8BiBu0AQFVxbscHtkCpj7z0JYSUBSp+cWAbxHCxe0BmaBGr9cT1vTEmrgLCB4C0g5QV2/KtOQf/4sGBY3MGtLnDw3LMI2J2xA7WptEF1EFATR9T6uikWcnxQlWFc93Jo5h5PZUOCNLhW09U2kfm0pdCV7dM0Vxgk/18wn877QUpak89aoPFcUcOG/0brPemHxUlIuPjSPluFtvKC3zrtt5a3G/fkyJmW8dXuXedksfxz13RN8dFy+yR1fd8w/H48vje/5u2mo1ti7NwQWtH2Yseng2Omrq7ZtMR90xxl3dvX1Tsm4vKT0bHbO21THec+V9qX7iFZRWlw2aJe39aMS2zou3mj+YjI2WvXN7ZzadXVM9mrR3sx1lhynnxbeed5Q93zLvM+d6RjkysuZJuTFp3+TMjDMdnZfLw2a2NOlcVU5N0yUXg2mBdNX7oqZ6p22OFfOsezI5xP7UPxt1u1eXzdZEeXfmdj/q73d2tszyRfkqzzPGm3e3yrgIPN8qJ2XleKIMzfvz3tboukeaV3c5o6TdnxTOO9PiqK68ua/fuC03TztnjWbmevSu0Cszo/622WkdDxW6tTtu5iyWtcpb6ofJ1c2k440P2+8b7MZoNk2+dapd23a5WGkcTeq7VqVwfNzu5dvXijnsFm/qZxV+0SadSrNe77bzh2bhfOejNlWVNsT0w5sd5ayNFXLcsJXOffPUvOZmqf7OPD3tHtYH9KxIWvWrRr2l0YxreY7LIDfc6+Zh9j476BkNg1vTN6yj45bfMTInw3bzpFTXldsPH1zM/d71UNcxreSM+0rhPb25LblDr3TqfGz0qNcejo/a+d5lL99q5rT6mXGx1bEdt11o+ZMiNm9Lu/Sa9E5s95LVO12iH3tkdHnbbgyzly1v0OvdFXOly0t/ooBGKRTcBPJkIkjrhDga/4B/8+rHuuPCXLAoyeD+UpblJyi2w5r9HXg9fSNkBRdqwegWTpXAG9KDaSgZDner151QgheOKF8gi4Y/seZDexAsxDQrRkA8wZQjhsfUxNzxZODsqg72dHniUU4uAPglF7zA2IjX4k4NBpKktDTqits0kHJBhwRm4uTsunVjn0eGMLdubH3YRrlMJhOMHdB8YQJJBqAtXu7SPCstlBNwddbBxAWkhF6jFqY2NDbuIEH8KuhsMGgwmNmgG1PwGcG6GMe3ln0X3Qx+4k5QjPazS8HVe5/1IUmq7Ycn9D5l7ig6aaTgTFadOymWSXR8w8Pw6AYlgs2rcldPWqn2i82F2kDxNN3Kw7UJVqoZNr6TGbsTczYnNlXlGc/wrSeWk3/u70RWLTFLzIasxOqyWOrbY3t1WdpAHtJTz9ceLl9Hgumv73LlvXDmzYZfInWr6PFYfU1o5iGJ98QCgCWQwxoi2Q8Sn1ntwd1vKoHm6PBA+jUR2Zj4XULB6XwgLcHHKvrlduTwvSWycGEPbcBdGAaCfId+asHnyL211WD/SNEdEI8R+9sG91vW2xjDYAaabJbcO+zhYfI75dmvidDxz864xYaXyz1pBbjEJkv0Q4ZUg86fLaFkqYDqqe+WzQEeOHhGWlD9QJr/cqKX9AIulwoE50ihqOuqUciWSaasqRWiapqxJjTuF5coLeMtvYFUCzAMivPIHEgHhDMSw3Ywz+eShW1USP1iLlL+E8FaQWWx1hoVQiqqYRhlvVBQcxhrGS2r5ouFck4r41xJqv3V1R1TUT+UUz9dCNufWyxPNadv2k1CyPzZPSXc9oKdZb17QPv4QH3o0vQ+4IgsqkNtw7yIFpcM4vbhiwrgX+bo5789UjNPheqx83f5x3GpBljsGWHfrMHU02f/Zs2+YJX+o4/175kMWQQId/ilKfEQvW9vxnF+twsd5McL9rfBCo+Xwxciudy/AMnlnhed3E8k9xPJfWWefV7Lz/1Ecl+B5FS9UsIFY1cztHyhkNnFu+WSSgjGu1pWz5L8PwvJVXZz2XKmUs7kNL1gFA1VIyRvlHcz+V1SLhiVn0juH4bkcl8GMHI/kdzf1dE/kdwPieS+67H+90dyf7NQfxuk8D3DmgvC6n9xXF+y1ueUOh3X/g+uRhxq</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_a0dfa5fe30174a9c9b95b75954d2484b\")) .filter((elt) => !elt.dataset['step0']) )[0]; root.dataset['step0'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtfet620aS6H89BaJcQEYkBVC8ipLm8yWZeDZOHHuSTEarTwaBpgQbBBgAlKlo+H/3PfY8wHmF8yj7JKeqL0B3o0FSsjOz+8WejE10V1VXV1d3V1XfToLwxsry24ic7gdhtoi822MrTmKyb4XB6f4sSS8DMiNpSoLL2ZiQ8XQ2mw2DXm/a9Tzf8d3pUb837PpDrzvYPzvJFl4MfyO9s46Xpt7tTfjbpZ/EuRfGJLXurHfXYU7aAOcTLCide9HEWlsm4E4YzxJAmUFKe+bNwwh4mydxQrEnlp9ESXpsferRPxNr7qVXYdyeJnmezI8tp9Ptk/lELXGRks3FhfFimZ/ntwsQSOrFV2T/Ali4IWke+l7U9qLwKgYuwiCIgNIsjHICPFwBtQzyScNtWgkUFea3DafTb967sOPr5IYKqkr6fvTi5XxKUiAYJ3njeJb4y6wJZKdJGpC0nXpBuMyOraPF6v1Ist+UaSQv2mRI/0x4cceWu1hZWRKFQZm1odROBpAkzXR92dR6lIU8XACOosgTa5FkYR4m0GzeFHhY5pA29fy3V2myjIM2Z5kWZGJ4GgEsUPGCIIyvmF7510g2jKGF2uSGxHkmCnsXBvn1MbRe3kbmIGtiIWezKHl3bN2EWThFxalW67d2GAdkBSU7jrO5ltNktWMtk1U7u/YCLNqh/8Nq0Qq1eEIXEnjVzRUq+BpvYMuPQv9t4OXefVosSjyU6OWcZJl3RSTtET16fXLIxpKTPCUk85MFaafLuH1NUkjL/DRc5BbVTdtbLIAHDyVwmPg5ydsZ4Hhz+2wP/0CxWW4JLqxTq9FoWqdn1t2eZcF/s2XsI6oVkIykIfTy38iPII1RAzsNAFhWSvJlGls09RHS6czSZN7w8mQKQC2rMacE5x0/CcgLFOWjvOE0mxPAXu/VF/M1iCE/6pYFMVantznJgM8HlSeIzJA2UonJO4sXRGk1KPnOdDmDsZ2j8AoynG1cP4v/KTzTYu7FMWM5Irn1BJVp7i1e/vnxU9DMiV6bK5I/AWUM42WyzChw48aLlqTF1BAwEU3UEClOvYxc0t7QspLZLCM54yOcWQzVOjm1HIFhSfBQHWfCUxlmmbK2SJQRicjZqeXWEJE560QkvsqvrbbVrZB2OypxQYyJ2M+zgiIr8kurYSbtNicmPp57+XUH5A4yK4g1K1yU5XxuuZwfqaVTrULnZREX584FMuUCC4xc0zrg5K06JOvAcjmi3DqssKtNhbkPLcw1FzbdVFj3oYV19cK4/p+nLeuqZU0vzJ32NgbTyX+U+lkYX78kQL3By3tLbumY/xNX+yhcPPdAt1Pv3fMwZv/iNyfxZ28h1LKgnuUezGev0D4JeBENwMi9ZanBqNmfhNnXYQwTQ4Nm/eMfTIVgqmqsmtYhIlgnlkvavRKvqOBKaJamzQUApZWBZYbEvqTEvixg8A8FiJKrRrXUA479a5pDo/CvRfKusWIALavbbBa6vZa0uOj7ihytU2UMwHwmTy1DEz9qDJN/bWV1eEbWxJkOWdRayRCjKG1lATX3Vg3R7pyh5qSmpicFRMHlv+gf1u2gtbDPWKd7WsuT1aJRqIAqA+iGpXqD7hRgompF0xdlgKhESYesI6tiOdQExyYPECyMtYBr6jJF18PeMZFUR2BpqlMQgx7jOqoKFMMrdvUFwVEWtR3m3U2d9ZDTFLR4zxJEjLPJe7dWqfRCkuWnSVV1QijXgMTJPIzBxkgLHQ7jhqQCpmprQx8XAWVBGu1aW6iIjoLoSrMpTGltpzJsbEAu/J2aTSJXTrGVWSBPXqEh/SpPweJmc71quxWTiJi6lCnmdXo1bXx2l66tz+6u8K/puvnaON2gfZ96GZhVVw8rUYJAdyQGr+YWIDpdtzuA/pnCEN0Zuv0u/L7C386wi7+n5SBVop1ZbndUyp5XxqYOkG1UaQFCnRhbEahWUfDPoxdeDg5TDDYQtAf8d9sCN5EmFRMlNHID1TSkBh/8cyJAuJEFaQcHTc1CS5N3AM8Bz8MLoSAFuTeM3BsgB7AFqTcyKT75JO/O31zIqVBIvuog/y+JnzfQungDvMM/YctyW5LNV2rkuqJajNEgvApzajy/SMO5l2JTnVNY+9MZ/WO34Kc7Gw6nPfpzNhvOHEJ/dn3P6fr0ZzDoDrsj+nPcGwynAf058vuD3tRucYLkaDj0uzRn6k+DLvvpDqfEn9kAQ8Wk8/WKQEqgcjac4f8otkf8IRlxzqbTIedhFMxGHk8dj8YD+tPvT52gz372xv64V3A2G04HAWMnmAbTEWN/TAKP9AvO9grufBJFr8CLApaGE5ahOS3gmczCq4rPEsCI831MngC+GOGo7kEjU7elZXEXJsxgZAuD0pdhBFvFgC4UghtnFFrSQq4gtAsDl3byNvKvG/3+5xg0aNqTPYMiQVHQEYeUG/YD/2tONtMcOhrNSsfidNEILmizj/NCT8+dllX+d9FSMlya6lYzPgjGRbPix6lC72CEAqYBqLJfOJy23uV9bvYr1kuFlIBSZpsw+877jjmPTbmjVyQuD3z3bT4DOZwUuv0+lQf825QoP6gRQcSuJGL3Qp5/a5ulaBi30pSbserKumhqJnjpmkMbPoN2AEfmVhE0a0FsKpCLIbAA/njLqrRdVaSmafpDNdfGmfn9W46J2L1/8zwsy1jW5pZrP6jpnD9E05USNTVCXffZ3B83d1V3S6fTm4hPXAE6vnwO40PeRLV3BNwZRsE0u+eh/fS+zX3fBn9wkz+40Td2sG2ZrtT/3Hth1mdeyFZn2XuL5jyxnHs3p/PHa05nk9ydBzfnvckamlNuutJsEQ3cVNu28PHvb3sYC6yQUPLwj6oCBm0SnEoqpTTd+sH6onlZ2wxJy1544CzkPDpsV8Jv72VRqr1KZJ/TsfZCNjs5CPCzBDZgaiXBR+vzw1uf+kRYEzcxyagI27Ro3KYlBW7u3wQ7Kyf1vadJrV6iE/vCS/Ps8e1TBC0ccyoX2eFC+Q0vqinQbN2WdWTOAcH2t0IMOEQP/z0CBdoK2eeQFKP3EIwj/Lf/fpjw7wD1pfQ+5RhzGN+gHw8CnXnQSnJnZVbw55ZrfaLFI0vrqcDO0yXZooUxufLy8IYUS4gn5QqngJl7V2BuLwNlCUK32aR4Da76FjgdHI/pUNzsZIsozBu2rZl6DEksVp5UFIvnmIwGBL1cICyUqeOdK4Qv5DEepbxI6Q6Jy5QsiJdnl8kMV6uXUaTM44bAn0J2Yh0chPqcx3t4lgM3LSsLA8J54FwylqWAYEWGAPgd3abDpQOwTRW4kBwduquV0XmqDiuG4BoTmcaWtZuoaubrTeXyYON9SqWgWrFG+4COhRXroJxBeOOUs0ilteTftRaCZgyJ/tTcbLVUZ9GKqVQ3+Unj+rlWvXOz7Vc6ecYMg0F4UVNdQwPL9TcOPLJZJAe7GPBHS+NDWhrGvSI7i2F38dYspaAZwZbiHsXBszgIfZI19Eh2yNLxRwZ6QlG1DUhiQD/nQwIdhMVaEiBZMBKYsIVmQc65jXaMfUEtGZh1SOpFtqxstIzOYpldCwTKqG2MPVVJ6ha7YL3YNMOrec5RvVWY2RcX6sQngE8tDpW9DReXdByytaUeid3Xn90ZwNfHajKJA0h8bfbGecEn9y2X4ZmobsOx2qaycGcSrldt5nuX9mCNpy+aVRZEquxRxHvUyG7MvewtCaxkmTftB7F5GSXJ2+Wiwq1Yv7G++ML6hOOGV3GSooNIR8sNrVPPV7U6TFWz5TTLwUajfbdQQcbbJV2rti80d1FwqqLWeY4VDpfx2zh5Fyvs1VgNEp5cWN28tIvssQuaRI/WnbnbdhDF3GfPtnedguZDekBVr+7VbKxIjfVdW21rm23uITXtta5zROTiTv50ZmtORRKRDknTJG3YPzJe5LHf5vOIcWcX3wXACniThLHwPZQNpo+gkV8tiF9Zpb0E48+7/THOw+gntuG7ERCM/dHtyS3Lo2BCdtL8RbeHhzDr3H4/zUh6Qzfv8H2wJM0IxRNZjQbYuGlIsmIjs2gvnn7uXHRCCfElFg8K6FRnH74V+7mXvsVt96eWxG/n1yVJb1+BPevnSfooihq2vnVbFj2rXENelhCuEImww2iFqSoEIJ2UzJMb0miaVLkqoU4QZlCJGG0PvTFb1t262CsM1cjyRzE4Dsjg16k3J9Im8BriCfsht58wZcw7u6fLMAoe8X3mX4dXy1RrfJ9GS0Stt6mKyuDlrtRVFmXVvCd/wniaJSyIjYEe/JJ3u9Jd9Y+9jAx6JZCUWIF9ykJFCihNkyHpvPUcpiadspYh4+AJBAwaBHQbOoeXEmXYlei+EmiZJkPeGiBvjZBZBHNAYADXMmQcNYpWovjaThDJyg1XMBK+ICluAykRlGRNkkvyNbWxcf3hmWQAK1KtA5rsleMUdmPaXuV2K7lJYbacsWMG0kxJAdiYoh96kLSkuotCIxzGW8myUwm1RAVmfo2+CY6rX7H5YRlny8UiSXMwgwI68zer29Wp3l2isaQWys6JaFrZlIVWSC6J0MnlW+ExwV+mKYzWamJGFtSJcWQvRgslFXt1S5XtiODDrZ7U1PaYYQFs3uTll7H/gh9k9UB8Fzv7Kf9K4lqvp1rjPMm96EkSZVq9k+hnPChF6+lelBmsOiDVihunCaBSb+M+ugwAAKcElkJnPI6JO3IhpwPWE9hU9Cc1sQRYyRETWWmayHUrf38JJA+oiKEs3OkfK8coipozagWeonCF5F4m7zTJgeZ+Q8Kr67wiuttdRXd7H9HdvofobjeLjleu/L1FdGXVJdkhYrNGFfmU42Nc7RWOwN9T6xYLvVvXy0cbrHcQkoYhSapa+DmVUwzWx4V0BmhP59r3wHPL5KW7RpD4yzl0vI6fEi8nX0UEvxo2A7WLY1T0s0MPIuLe71I1D6wuHo8Quw8V8Gsq2QKetocZXhLqE4qL66RklSu8cqp8FRVyG3Y3oCwaBgm+45qeTfk3vjFbiROXMXswP3F6wyZUTSNpFUTMnC/LDfByyHnT0pVxf54RhfOsWq98nz8UWIGfS8cBpM39ld3hfHe/vjO8sr7M9sTQEyCPppmpxCJT3ZhQIHqrDYg0s8KwuaFOmXyraxmVZuAb2umWYxQIbrnHzdu6cMzr99yApBvgnwuSG443lX9MfLew1FYpwlYplJbldNx+c7JrdRSeMPEAD/AdlufytgbD+XoZHhIyqE4YG5rwPnrGDh/N5YM62yrldPo7tMiGFm5jdWgLI6fsa0sQZs9wugOM92f6CUoprEsHYPAptRFYOW9RkDg4lT0EeSD+cvNAPdE55C5EOZ9I5ghqIBubHG1PxW0Jf7sZng9j5dRv4bHqiMDonOKhpTtp/YRbBQy2SK0FV35KDgLgU8P5vJDXRVXpRKDxVLGG+QBg/Ymu31jH1ieflNk19Aw72eUIgja33GOH+15l3U7RUUUHzT+ZMRXceLFPniTLOJd176FGFbeIhG6BfXMgNW5h4GBqaedQMGEQqbC1Wq0YbKr6liOCzMfZqWSo4XJxZR1W+1RlU62GWk1Ndirv7a28Q9dUuNNpocyASqNGaM3N+DULz9rnFOytt/VLx2ttuNV057QQh3E/glZapSx53rD2qrs2lHFgVa+2D3GiqP+zQWVxiDo4LX2aWn2t1dbVZm1Fca5UXV1t0lXlY7VFT1ebtLRWR1dGHV3V6xgKCTXULKXmJmSjeu6sLor5utKVcrVJKfdqS6ibrNV/DEN4Jwpj8jN3StzJBsAsT5O3pGYxv47yE2+BwNmvSy8lW6H/klBTy57jAq/9u06xexsnNl5Z054TtpBeHBVro0nWMox+22CK3C9lHXQvrD/9Cc1U3JuwAUMaV+tQ6qZU0zzq1s6j7sd59OM8umEePftw8+jebpOnWzN5uh8nzz/05Hn2ASZP+rccDCtvVCG5EqdoxOSd+K3uV5IycEI3BTmafClbsFAXaSsp3TNKhltOtkfG5MhdZZFWjwgWkmBbE17gshbed3Bbd8yB+snSjUQ0+EADMGIiw1s+3InhhL2MdVuDpQoE9znj9TWIz39i6pnckcsYf3mP0gVA7qnD7ZkyRUnRbQnLcBuOHMvcbIWJg3PZMsqlmPcHi6yIxWFGpIz/QwfdKZxicd7UkDilN7ln+EZBWqtRFNR2zOf3TVXuUgjV+E0pfRznz5iz1G5XK75pcamAocxa24d5HtRLgmW0zFR4evtTgUO/ZDy9fuXn54LepHLcNUkD7RovGfGQ813Zpl/eQ4YEDgxrPqKjYD7vHwzprFwsUodLg1rL4+/vryLQ82QRwqdRRW4friK3O6nIVntV1xEZYaOSVGv4MCVREP84SiLuWdPjoy3LGOZscXbKoOXFxgXK8kpMjOdL9wSVW7AwdBkHT8DhDGqXAYPwxm6qdymGMaUqrcvpKyqs4PvTp5QVNLbiJxb4StL0As9iNdKWLka1J0bQYiVyB1i8GvhrejMw9ebF3cA10HnqxRluNv8+Da9YACBPFjAGzOrow8T1Ik0WJM1vG3Y4965IOyWo/WF8hVe80D03IKTAbu5AoN0WF5C2f0uSORJwd0TE28ba9CLkDCwTitlbrGwhbtYcqqhf+17kN268tKGVi1bzZ3fyMvF6sRKnAmVKRUvsRoqB19AS19PSgwQoMzAPbKEsDH/nRpLBFTnRe5JROCZW7erisy50fsfvt1BWUevP7ioD/5pe4NjpkzmOs1BfUeEaen9NFhK51U7kKrsYouhbb0oieXMHX1Wi6S+0ow98bsH19zn0CwpDQ2p06TzCT2X9nKYIDcLLr15hR0LpL6R4mgzFbkp+TG/bRjin40INpDuTjViow+zqI2kosqlo8No2ly7LNpzOAAfXaiM2UeNYhtolmuI4yvr33zNE66Odqt48apokIfcIcWWzUWSRrI71nZB7LVL/09uLtxQj5DrO56Bvn92FqH9U/cx4fDyRaruNE910LSOUBuYwCvqEDfLCiaAZJlYQ9hHewI4yk0YEy6joXMsLj7YyZan5RQ9ju4vkzPXvvpfqn6xSyqiZJjmQx2GzPXYCcmUbaRvGZa5RKQ77xmJSZf7YWeNMmtq2Hq7599bgqmG9UYUNyqnKZAft3FMtaFDVbx+sEiX61t4lge6gQhXoaTn61wMJLdgEA2O/WBuysb3rIRXVLZSynVCtRCVWtFJTZbn3l5QfOkQYKEiTMYO5o0DHUj1aeNIKkyjtdbNuByANvDxOViSrt+Dfy0coC+j4kZdl34ZZ3gGLBSzdeJagKPkzDIXldJ/wEAdjtxxu3CbKvSC2q/gVZ0pTfG0lQWJ9l2rjsyS2dIuzVhan0Xj97/Fnd2UfWZ+/1jbw4AMYFdbqCqUvZ0i9kiHzKKpls8c0bC1XbMWpSokDsD1i9dneShW3qwFkOVnIaxtmUTBxMpRtUrMvLFuTE9ObB8qJIRdyou+i2FpmvZg4QJ2YRHatmDiALqYiuephob3ac/xrvF0l4xNEOUNski8jWpWdj4FTyHkC4w0+VEKfqyA3uWKwV2QBAB2o6hXJWVIZ+tB0qx5wT12prt+2zK+Y0Ckp3VuKC9XE//Umh7HnK7y4AQciAsOl0I2WSSh6v3oIcnXQxShOBBQ+4JDLKZbjj0jp0I10dmF/zNmBOLtydh2v+oahFg/y46U2nT49U+/iJTTyCoFag+8XfBpX+C/K3oX1hJKQB06VeKFRxb3h8nkZNKeqA8GOw0CJbBwIpGzW023XkEM7ud01ZPHubXvxbTVT1MpwoEomQfJHeZ6GU5jMGzZt4ZbctFogbpbgS1sfLsDHKRpnbg1ks1MvoMRo/vcEH89q60R4dikAc759QP15ejJ5ljSrcYzimabaDlY1wnYSiCDMJPIdDFRYT5FqT3QetIn8w/GAg1GVB7wPSwO6X7iu6n5ohHaO1lUpcQuseMTpA8ahBc06E7MAUMNz0jXrq+f0XTeYOLSRDgNAuHGKxvqxvWAeeoxPeoXx1ZMoBGZeKoeCpSWfGNyil0Jcn90JStxVaRekaewFBPXauCIkYgCSzV+zE6QwcUWYTsJRl3g4SIf6CJoDVXBtgJdjrWYXCaUtH9TGWlA7l8tQI7VFmGwJp8AX2ndWio6laJdvbWNXs/NV7lmgokGDFE2LZmHwvdF2On20v9xOl8y1SyLep4aWqic8fltRllIIXFvw4JR/Ld+zZ9iL8mDSjho40XeVKHo+W0ai8oL2Jk0u/dttesyJFgi4nWerPCVu1FNTs/y4fLlDQNEqt0Rh9Et5kCNPFiYsSC6R4EPBoXI9Lk/PFlg0o8SjnwqmeFWyispySlz2LSGvDe+zbFp2kPSAhn7bliISeTjatOBQCXexBzFpBK4oAJdWJPrwqcS3auhLQ6cqQXnoTPlq0DZixQqCTI0HcSRyLKWkt9ZOpePEBNTQfKFbhrJyXwvT2uUCphmC+V+nyfwVt0zNxwiV3QF420nwYsWuz+Dr4TS18S6Mg+RdJyA34GHQUimQcjl/DQy+dqVs7tGLca1DuSj83Fac+v7PogB4FLxZZvmcxQC1gmqp1r+442fZi9VfqdtHHylJM3aev6Hb0IocZCyt7ipBV68OYivoX2qVaFqfS09qnFYuOHm/JV3zFZj3oukt88SuaSa6Zol6KNdQ3oT2pbEl6zrVlgVn6GO0QOniKtnCUpbQGH/vvNy/lvRW7zF8UwoJQvWiYa5Wc0R/jrmN142U0IAyfaX0s7saxVsHCxyCtLBIxN36SvkWK5vfGqOFAPxr6jS2CnRpNjKNB/JspVVciluIH6zgauChrtR1xYmsD1qoldwwuBUXyEUBW2eWyLPbnCBZgG6u857ScpmfJlH0WLHJ7uikayoBnzOkHLSsKbn2bkJ8yNXGK5a8OLfXehmyI03LeRbnyU8hede4M6ADzSjx30JKTDxQopLgWj30XpXnPFlmVDNQpnoMTVx+hPtMQXLyhlMMbbHK/a1llR+/KGOawDScgC7cUdYt+WO9NPCbxES6QVLz4uoA6y6DLk5DXt5o1yQWm4+Kmu1SKBV0PXtsLVqx0re4m3TTLD0QXbOGVikj16yX7UXQAx/bi8DmUBfEzDdPKmXXC3jDlTnNyQ6KoItaCk6AaQQQAz0PLSYSiIWygjJTTZZ8UBAwI38jogQaNk83odMrxcruAD1d5uPMcqwvvlBEJgMfaMDM9ZI4Vh1ETVp864Ot3HerwXDvWPMcd1s6Vqslqq7pjsF5M7Jp5GCnog/qiq6rrCKRdU1L/VK01DfCOd/QVL8UTVVAy231jcGb13ij/XZzW4n14vdsrF/ep7Gqw8s92uqXe7RVuThu1x4B2Wn6SphFoM9eO04xO00wWxmhcbptc+hLcdTggTOpwK9Op9VTMQjw8MnxspwVWaET/UbVy5vqDbCbuTDf7FEEQNWZx36CGSQ4xmC5po1b7kPWK1O57sT0Z7fJylJfzDIfKVJPVG2TSc1BrAqa/i6A+YaQnXT13TUhkUlXxfjoRTmUWw1sBSTKvV/QxWC4zQ5LKblCbH69/FMy80Bx9JBo+RZ1xWhv4tsLZT6jrb17s9HSl2VSBcR97Rq73DWAWQPhvgW+ox1d9uJ+OhlRITTRD0JXOTqzettq1z61err+KaWe1LCLt9lUrrNR2VU+D9QTjKajfrX1OLHaWytysK0iZ3UVCeN7VaS9vSKGxUyZxO4+cPXHw5Zeqk7lnR7JpR3Tp7i/yHE/DOFah5oOf6mNeQ1NQgq0EsdlwWapsL/pyz0fsjSTL1u2l+G62oWXZeENOWYPuKyVNTHTLopNDWgMYFQurGWBqu9j8pQ/2/PhrqtVbxeSLxo1bH5lwQsOJFIMcEW4WAZliRP1eqbqFXq7X6JXuUZPvQSvem1eJZ+xkOu7xeovxqvei0QflGDvNvDJH++aAC3BLRHH0kNLoDl81tVOxhYHix6wi7T2CI/rewu7Dqo8vbMRbE4jjqzNbacz7JN5LaxksYYx3t7RVj3nyrqC+aiJGRiN41lEDwPbN0yla2E3nzYySKz2HIUOibUqPHO72+nbO8V3t4Wv6+q8zLFA2kiLlXZkY2/T2bDikJfSCTbqhwxSpxwyTM0G5D1t0fmvOC1UrjHHnK+iii++o8bX1VqQlY0jnqTXvSvvXa6AGQ+/WBiUxy4ArmaxnOnSfV4wE4nKipsG1mT+upb+xs3bFWgRXGn3N/FcerGuuwlO2f7uk1i+WKdatKLvrglQdZf0CWGt6wObAf44KsHq+y/TCh7r+ZerhJj4Va2o27JUtGX1Dn9+xveOPxag2UTH1SRmxxmu0z82JSL0erK3blIrLL8OqQnwMkny75KANJqd6yTLwducxVlHBKDEjYzwc3JyCJZzuMjPTg7zlJDMhymgnS7j9jVJydkJbnC36A6r0/1ZEgX4msdlDJT3z06onM5O6MKShZbD6b5/Tfy3UId9I85lnlxdRYh6SJFU8vS2j0tvOgUveP/siyifyNl2nOQ00z574606VAgWMA8QChn00eMSUIDwy+MbvRZ4jjV0rf/+z/9oOx1n5Fr/7/86nf7AOv/v//w/kDI+akHefzmd0ejCipP4N5Imx+5gS+kiW/zDxSVXmUKTFaQEJNhX8/DAOGhkcOmjhoGWbMyHTNwGyvNE614Weqkha89s7J+9FFM904xOpyP4NmoFVRne5tAPotCnfeIw8XOStzPA8eb7Z8XbWaxfUeVkX6ibkyI2MqPvBWzU3QkD6wAPr5IkFu9rcBL5gkcfcNhVXhVp2DmZL9BuQTokTUFgYCHiyZPSlmXPI/zl1fffdagn3UCCHX5AXKfH6m43lUEDqCFK8cCIpXa4julxDMZu+bIGDhpbeqOoi2hGpUr7G9rlTZaA/tzt4+bI/WNr/ymLvxU37ID7gFFDcJgtj+4EaHasb9CCPWTpuKeLPYtAI477LWtfegwBKUY32bMX76JHB98Ofhk97x5Ez3xyM/omG6aD2/TXZweDP//1mxfzo+dPhgc/PoquDv/29U/LG/et8+10/KvX/2H889Mf/JvBbz89fjrMrruPDv/+7e23L65OT4ui6HsNWBLvypihvY2AuY9++Opx9f+MjnQ5A4De7Xsu/ONiAQ786IGzs18ezoWU87t9PJeAZAEUcmkXpp8AZbmAg6n0UAKkgje1D92IkrqA37d1tJwqLWcLLe00FBLEZPWGJFopIabyLiWkC/M8JNKxzBmPxuORMxgPB0fuCPO8FZZoyuJxWshmDuJ+cas1xXCG3d7Q6TuuM+h3+72uyyHoZdd1NMXDz1iH8/GAnTc4oi/Sukc99tlz6edgCDkD/D/9HMHPIb6tyz7HQM118BFA9nCvg99HY/jLZe/0YnHuEF+a7LOELiR0HRjN3TF7EbeHJx+whO4Rg3C6ePaB/sUKGSD6GB8mdBya4OIJiaFDUzFh2EeWsaQRxTjCn92j4huJjXsFOJY4wEoP6Ju7R0CthxRd9+ICFVA5GwAiGjI1rwuxM80Sbc6fFdwXOFS9sPPsYwhoI9i5CkKvK6F9D1SuVNryoTDUTJEgFHVbGfB7SyHu+xdysb2u/x5bMN6rcCxzfbGWh2A+3MJPmFwrE3pl9tdMmE1T/T+sp3R30rH15MWPlsOJfXGVT8ylMAZwiqdGLQz9LHZxjPYI2bfCAG2t9LKYEYJB0POGgx7xuqTXD4LprOcOiTP0p2My9f0ZZw/+Rnpn7FUgsBfm6KxcKtweg4nU6CiW3PG1lzXOmEXTMdl4FOeYmoMkwLsVmnSyjjBi+alH/8CEB7Wjpf8+ZqbFi9/F3DRkGWVhalUAj7xFhpnUajUohkbjU6Pl+r/Wwt0okXqV/p0NYirAxvk57ms/6juj0dhq4QzYG/VGgyN8bhY/nG63O+iynIE77vYH3UH5lO05ggwGg77jgFyB0pE7HB6NcdLBj9G4B73qiCL3ewMow+lJyAAy6h0NR2Oc1vjcO4YMVthw0B/Rj3IitdSSB64D1IcMxO2Oh106QSLZ4dAZ9jhyd9gdw3Q1uoAJhXmptJdwhWjWSP/9hpfxqAuT4HjodP2gN+vPpj4hR7PhyDkakWFvNi6Hlz+eA9nts97VY73L7bi0c7kd56P3+NF7/Og9GrzHwyfeX25+i34ElK9eHPwwHrxzv7q5eZI40/5T//HBodf3pzfd6fVfng7z7quDw78v5o9eHP38zEmd5c0Pzw/J8Nnypp/9+Fva+/XWGRxM3750P3qPH9p7hBFs6PZhxnRGOPbDlFN4j8asLd6j63SH4/HR0RE4huNxF6c2xXs00vzoPX70Hj96j7+39zgNxgOvNxv5M/+o13NG3mg4mBLieSPfDVxy9Puadx8dt3+GZfk/22srJxvmLo2Hw1FvOGDuD4x3vd54yFw4d+z2+uOjnua1OePemA7I6EjB2O0MhtzxcobD4bjLvSr0scBx0xyv0WAMIy4MwzgJOb3ByD1iyGLOosh9F3w+aAwdGSYGRzibThd8R5ynsPkcdwy0RsxrOxr33dGo/95e23ub2OyinWcBveWztApnY0LG09lsNgx6vWnX83zHd6dH/d6w6w+97oDaRDuHm2Tgrc6jDLx1KLqYcE8g8VMw/o2+gDDXUVYciINrBvp+pxDnJWbvV1CfRAnd4SK+Oz4m0GLogz2TMiclMMj65Ocwv24o6AXRWepd8eOz2raBp/zzaw6BdRDQytKzIIz5hpP3ReNqJ+/ZU8DiaGxBGT75hoXHt8+AuMDGNzEm7BHpZJn65Cm9c6ZGhp/iXLJvHVgaOr/RSRZLQa0zC9NMlE1rBghlbukbrdWWMAp5myv0/wGGAKA9</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_a0dfa5fe30174a9c9b95b75954d2484b\")) .filter((elt) => !elt.dataset['step1']) )[0]; root.dataset['step1'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Restore the checkpoint back to its `nnx.State` structure - need an abstract reference.\n",
    "abstract_model = nnx.eval_shape(lambda: TwoLayerMLP(4, rngs=nnx.Rngs(0)))\n",
    "graphdef, abstract_state = nnx.split(abstract_model)\n",
    "print('The abstract NNX state (all leaves are abstract arrays):')\n",
    "nnx.display(abstract_state)\n",
    "\n",
    "state_restored = checkpointer.restore(ckpt_dir / 'state', abstract_state)\n",
    "jax.tree.map(np.testing.assert_array_equal, state, state_restored)\n",
    "print('NNX State restored: ')\n",
    "nnx.display(state_restored)\n",
    "\n",
    "# The model is now good to use!\n",
    "model = nnx.merge(graphdef, state_restored)\n",
    "assert model(x).shape == (3, 4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "    The abstract NNX state (all leaves are abstract arrays):\n",
    "\n",
    "\n",
    "\n",
    "<script> (()=>{ if (customElements.get('treescope-container') === undefined) { class TreescopeContainer extends HTMLElement { constructor() { super(); this.attachShadow({mode: \"open\"}); this.defns = {}; this.state = {}; } } customElements.define(\"treescope-container\", TreescopeContainer); } if (customElements.get('treescope-run-here') === undefined) { class RunHere extends HTMLElement { constructor() { super() } connectedCallback() { const run = child => { const fn = new Function(child.textContent); child.textContent = \"\"; fn.call(this); this.remove(); }; const child = this.querySelector(\"script\"); if (child) { run(child); } else { new MutationObserver(()=>{ run(this.querySelector(\"script\")); }).observe(this, {childList: true}); } } } customElements.define(\"treescope-run-here\", RunHere); } })(); </script> <treescope-container class=\"treescope_out_5eb8fb6116ef450e8276128216cc9262\" ></treescope-container> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_5eb8fb6116ef450e8276128216cc9262\")) .filter((elt) => !elt.dataset.setup) )[0]; root.dataset.setup = 1; const msg = document.createElement(\"span\"); msg.style = \"color: #cccccc; font-family: monospace;\"; msg.textContent = \"(Loading...)\"; root.state.loadingMsg = msg; root.shadowRoot.appendChild(msg); root.state.chain = new Promise((resolve, reject) => { const observer = new IntersectionObserver((entries) => { for (const entry of entries) { if (entry.isIntersecting) { resolve(); observer.disconnect(); return; } } }, {rootMargin: \"1000px\"}); window.setTimeout(() => { observer.observe(root); }, 0); }); root.state.deferring = false; const _insertNode = (node) => { for (let oldScript of node.querySelectorAll(\"script\")) { let newScript = document.createElement(\"script\"); newScript.type = oldScript.type; newScript.textContent = oldScript.textContent; oldScript.parentNode.replaceChild(newScript, oldScript); } if (root.state.loadingMsg) { root.state.loadingMsg.remove(); root.state.loadingMsg = null; } root.shadowRoot.appendChild(node); }; root.defns.insertContent = ((contentNode, compressed) => { if (compressed) { root.state.deferring = true; } if (root.state.deferring) { root.state.chain = (async () => { await root.state.chain; if (compressed) { const encoded = contentNode.textContent; const blob = new Blob([ Uint8Array.from(atob(encoded), (m) => m.codePointAt(0)) ]); const reader = blob.stream().pipeThrough( new DecompressionStream(\"deflate\") ).pipeThrough( new TextDecoderStream(\"utf-8\") ).getReader(); const parts = []; while (true) { const step = await reader.read(); if (step.done) { break; } parts.push(step.value); } const tpl = document.createElement('template'); tpl.innerHTML = parts.join(\"\"); _insertNode(tpl.content); } else { _insertNode(contentNode.content); } })(); } else { _insertNode(contentNode.content); } }); </script></treescope-run-here><div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtGgtX2krzr2zpOVf4lPBQUPFxvoC8bNUqtlp7e/iWZBNWQhI3GxDv8b9/sxveRIrVau2FnlNkM7Pznp2Z7K7HexbZVzgjxNMcl9SZ43D0D3Idj3Lq2DnEiIU57ZAdZDg2jxu4Ta1eDrUd2/FcrMF6t0k5icsfOeQyWLGox+Ny6zjvubBqOzYsN7DWMpnj23pccyyH5QLUHdT/1bAAAPajOm/mkEE5gNmc2HwHuVjXqW3GLWLwHEprTUHEJvEmoWYTVlJKRmxjc0yB5yFa/494h3q0QS3KgXPsc2cIG6c2Z9T2qBb36B0JnvbZvd9NBOrZHaonznwbaDJY8zRGXY6EfHsr2HUtqmGhsYSjcSKkZwS3V/aj0djePigU6Hkc6cSwPbSHeJN6ikn4GWj72NFJNKY0HY8r8jmIRjiqu8QWIqua2FUgffse9qSCbd0i8Nj2LWsnoKAAmzXHsWE12nVYK4bGeXAuYEk8mljmVBOLLmGGw9rY1ohiO91oTNoXCERnnqB4gLSL1tMx2IcaKDrFtWIR2+RNtLeHkgJkLuuMcJ/ZoHdELI+MGGv6tuBsemuvSQ0u+JMA4o97+PcAhSh4la07XYWRG594XLVpW5qrxHCbRAOdxMQeOzOEXN9rBmrcCZFxQGIvEGOOlIvzILgIDMkd07SCqKzLyAFvdcVeYoVYfA2RDjh435KCO/lbaZGeUHqERQRDfWBFs7DnfYTg7O8bjQz3rLfBDSMD4vcx0Ce4v/Tx/d1EWADotIPkhnuRyfQRQRw3QFJyuxdJRiB0GZ8FcWxgEZRhw6N5wRCugajAGcgegWAM0pjMI3XcaDDSkf4j08r77FYaJ5MgVR9Ac9ptQByDwPIjhJ8CwTnb4dFc0+kQFguBnwSvd5vErpNbF0xOdImqGI6l4wZIYINouSb2ovsWbhBrf/JJPZAzIKc1idYieiyG/hNDD1MFDYJb6mMQSfgYxgjC9tsNwsYBtrdSmewIwBPZzxynsZ7KpDICYIK9B44E0FeYFACtU8+1cG+Q+qcB0T6SWsjlGgSyChnjQJOfnVB6QfqPp0T+7x8TYNchLWrLQ6FhOeIceZCmtOYsZR2zlkewCZ5qz2I/kzWHPAjUcKQB/ASH8hzLoZW/05mGtvKa7E0iPchk9gWYFHYUhH3mCQO6DpzmhIXQpd7zkZWhIAnFZf7xHvLx56E6Eo+TWz5LRaFe3aDM43XHrgv3DwmteaGkpDMimkJNhZ7MfmDxaRaFVG3MTKi/AjZkQN8/kRqkNLfX8DmHwicsAY0ehzltBEWmoECRUNSGA/9NUht6ZKomXjnC4BUUW6jWazccy0MnPhfy6qgQYMK324PAiHdJowX1rUT32nC2NSENQ/Vpc0Cn2CP6sCp+T5Li386smwfYsmxNKtukPS1lEB8hUoSnuxGm0sVeXYOKFhQ7xMcGnzhKBnl6Hs0pnEmS46pHHcyi8biOOY5jGwwrC6PY+LIgIqo9hu2BN8ttUcpDBDQGpXzc8fnjRBlyAIahRH83yYkkid7Rtuswju2ZvRvMacGBL1ZGyejH2h1DG9PnwMz3iqidgDG9rkHlrTNi91md7G1gz0nAZ6s3hqHTP0gnQlXDlhaFBgyq/5R7KwtGxeNY4A/5/WWcNKDcgbwbcKI7HGQXXIwr78bHlg31dB06UYPewiYTYbIlwwSaB8yA4S5mNgRefZDYB7YwDKyl1kMAXai+/xl2o6zffIrs1VdSfymeVGRaHbW8OdmwYhY3GdYpmC2aWs/oxFxDDni0SVASuMtqzbXAw6FqFvlCLqG+lmdYmUmsz6b4CZMPhLlXZH0LSrKwCynqx2Xe43P4wxSC4k7SmCizH4B5Dj7CSPQFnQSoQyk9as/C1PIw+PPYbA1NDXGUydYOLcjwtJaHpqgDHrDAn4z2ovIuyP6EUecbchGqC6loEV0uQuxHQj3HFEsMFtA7lTHcUwzmtKO6o/mi41VEcvWUDrZ84kVjMcVz2iQqU64YSYhvJSiPxDhiwQIpsoJiKDYcAHlNQriYEpEuKtRqNSFNTayJmY98qDACImuk1rO16P/+2y/KNDJI/o8v0IIzQ1ASemRtbPXXuv2B44YYKHhMyyGfWVFRLeTE80TXMYz0TgPqkezGmp7cLh+Zal6Vn+qpqjryr/xZF/6vlFS1qM775NuqaracD3q1mC90v6rq+dfCoXpUzRfUknlbrXxsci9/RIm5Xjq4TH+sZr92aq5PPx1lzlOHl9WzL0edi6M7/qlXKhVWL8zWOc0fJJv04NQ/LOrl62SlkTA6Vd29+ZBt3lxQeuof2eVmxfjM1c/Z/DHbUEtVu1XMap993149y9xoXqvbMUpW4ubWLDpbZuOwW95KVdSErZ5lPjJ2mDpbNe+SZ3pSPTRS5vFmoVu+TptJp+efbW62i6lst3K5fWKaLjlv9TZItXGX0RrspMyxap5Wj7sH2Ot5p361enlRLHXVT6du9av+OZFYNTfPNy/XedL48OlG7WRgz4/q8aZ61FXb5t1ZbdW/qpHi5W3ayGp3xxtnlV7Gz6sf7vLXbsldp5XTQjF55X/aqG3aRv5jsVI6aqt0datTTDftVHNztfGle3ndrbDOQflzwb42ikWTr55oV5a1mdkuHHbzW83tjaOjcm29fKWa7WrmOn+6zc/LpLJdzOer5fUDc+Ms8VXrNdQy2PTLh4R6WsYqOSpYauWueGJecTOb/2SenFQP8i16miGl/GUhX9Jo0m0yx7XBN9yr4kHqLtWqGQWDN3sf7IqOS17FSB63y8XjbF5Xb758cTH3aldtXcd0O23cbW98ptc3WbfNsifO10KNsnK7c1her13U1kvFtJY/Nc5XK5bjljdKXjeDzZvsFr0itWPLvbDzlSrRjxjxL27KhXbqosRatdptJp29uPC6KnAUQ3Liy6Mr0q1XRB3yP/hvGP1Yd1yowUYhKefUiqLMgVgLYvY77DV/8teUg1NZJgcVPOwN7mFrKBoU0pNjbQjBc0eEL4D1C22x5kF6EFuIzkGU27iLKUc27lATc4cpsLPbcDDTlS6jnJxDkx0d7QXC9vcazU6h+ItGxtoKMTUFKue0TaD/iA7G6jN4jLShR5hBvV9D6WQyKWs8SL5Q7kVlgxxOd6x3iIyYE6OBQQYTg+YIeo9KmFqQ2LiDBPA7mdmgrrOhPoZsTEFnBOui9Vkd111/AvyD2a9oowbD38kZ23RFGtnfDc71XWq7fv+kiciTvOHcRkI36R/68DA48IEJiTxJd/Kkjez/ZXHBNkDMh5t4ONUtRPYNC98qtn0rehpOLNpQBnsGXzWxHP1nN9GXanyzmb4tMu/51MPxeSwI8/42vbkTdAyp4EefgRx6WPvPquxQGVdGbewKcuyCcOO9lUfGsZzex1bQsMfei3xb6cu68j2C5Lm7FxlrwnPorxvf4TtjYMHCDpoZGsAxLz0ZMmUT/u6reX/SjL/Wbg2KvRcx2qtERgdDCQWceFOh8aW/HoTIK3nPtxWh/IW9aAD+it4k7f2S7jHnlVUkxEFGz96jB70otK98skd9wgy3h0shGXce888Q1cGbxCCA5AMURHUI45LV4Olf5pxwC2f2BQNDEQ70yOiQOM8YIhMeGNlHUK0cgEuIPgxzOSdD2cwWcgyU+OwR5iVop3cH2jMTyW4yIbSfaPhgtES9TnTKpd/XFbEeTyrbSjLu9tbjov0FLnsBPFgrMbKW25vibs7X2qwRh1MF4H1p6flKnBdycmywF+Ia/Tfbkf1jMOICGl77ja0ghXysGSTSs9oh+IrNLxTGMtcbd/6XPf8Xd/kf1o4twmxivU7Jv6wih24RmOERfjRAWFaSy0ryT68kB77+uApjAmtZTb6ZA/X1rf2kivKFhy3zE9U1hPvUQLGJXXIg+Khx5mv89Y+8R1ask2ivePx5QpO/2va/i3kUKe1PGilA/r0mqMFV5cj+xh+QKMeU/C35/UlGAvzX7E2mrfLmmvBQnaaeaJPU91/Smv9Rjv+aqWlxl9dne6bpA912Bye2BI4OCncH8/V08CP21sNC/7nSbgz5FwbEG1fu6xRSzzH8e7OKf+mhTdjX/VtN5y/z/v0ZJ6YBS+l/0S2J9GJWSi9vSSxvSYR6z2PekqSXtySWs+1/xWw7/RPvztPLWxJvsMz5DSy9vCUxodGFW6X08pbEWz3/l7ck/rQq8jENd3p5S2JZSf6bKslHvzdPL29JvN0D9fWtvbwl8aJ2flzFurwl8RrmWfxVZHp5S+IVLbTgLYn08pbEC9ok9USbLG9J/N6paXlL4pEW03+utFvekvhtC6l/2y2JVx3azLsl8caU+DJv30MV9pzuOYTUaWf//8wFOMg=</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_5eb8fb6116ef450e8276128216cc9262\")) .filter((elt) => !elt.dataset['step0']) )[0]; root.dataset['step0'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>\n",
    "\n",
    "\n",
    "    NNX State restored: \n",
    "\n",
    "\n",
    "    /Users/ivyzheng/envs/flax-head/lib/python3.11/site-packages/orbax/checkpoint/type_handlers.py:1439: UserWarning: Couldn't find sharding info under RestoreArgs. Populating sharding info from sharding file. Please note restoration time will be slightly increased due to reading from file instead of directly from RestoreArgs. Note also that this option is unsafe when restoring on a different topology than the checkpoint was saved with.\n",
    "      warnings.warn(\n",
    "\n",
    "\n",
    "\n",
    "<script> (()=>{ if (customElements.get('treescope-container') === undefined) { class TreescopeContainer extends HTMLElement { constructor() { super(); this.attachShadow({mode: \"open\"}); this.defns = {}; this.state = {}; } } customElements.define(\"treescope-container\", TreescopeContainer); } if (customElements.get('treescope-run-here') === undefined) { class RunHere extends HTMLElement { constructor() { super() } connectedCallback() { const run = child => { const fn = new Function(child.textContent); child.textContent = \"\"; fn.call(this); this.remove(); }; const child = this.querySelector(\"script\"); if (child) { run(child); } else { new MutationObserver(()=>{ run(this.querySelector(\"script\")); }).observe(this, {childList: true}); } } } customElements.define(\"treescope-run-here\", RunHere); } })(); </script> <treescope-container class=\"treescope_out_16a3675ef842438cb85db5d1411974db\" ></treescope-container> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_16a3675ef842438cb85db5d1411974db\")) .filter((elt) => !elt.dataset.setup) )[0]; root.dataset.setup = 1; const msg = document.createElement(\"span\"); msg.style = \"color: #cccccc; font-family: monospace;\"; msg.textContent = \"(Loading...)\"; root.state.loadingMsg = msg; root.shadowRoot.appendChild(msg); root.state.chain = new Promise((resolve, reject) => { const observer = new IntersectionObserver((entries) => { for (const entry of entries) { if (entry.isIntersecting) { resolve(); observer.disconnect(); return; } } }, {rootMargin: \"1000px\"}); window.setTimeout(() => { observer.observe(root); }, 0); }); root.state.deferring = false; const _insertNode = (node) => { for (let oldScript of node.querySelectorAll(\"script\")) { let newScript = document.createElement(\"script\"); newScript.type = oldScript.type; newScript.textContent = oldScript.textContent; oldScript.parentNode.replaceChild(newScript, oldScript); } if (root.state.loadingMsg) { root.state.loadingMsg.remove(); root.state.loadingMsg = null; } root.shadowRoot.appendChild(node); }; root.defns.insertContent = ((contentNode, compressed) => { if (compressed) { root.state.deferring = true; } if (root.state.deferring) { root.state.chain = (async () => { await root.state.chain; if (compressed) { const encoded = contentNode.textContent; const blob = new Blob([ Uint8Array.from(atob(encoded), (m) => m.codePointAt(0)) ]); const reader = blob.stream().pipeThrough( new DecompressionStream(\"deflate\") ).pipeThrough( new TextDecoderStream(\"utf-8\") ).getReader(); const parts = []; while (true) { const step = await reader.read(); if (step.done) { break; } parts.push(step.value); } const tpl = document.createElement('template'); tpl.innerHTML = parts.join(\"\"); _insertNode(tpl.content); } else { _insertNode(contentNode.content); } })(); } else { _insertNode(contentNode.content); } }); </script></treescope-run-here><div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtWgtT27oS/itqOnNILsRxnBcJj7lOyIsWKIQWyjlncmVbdgSObWwlIZzhv9+VnIQ8TAothXIamCEgrbSrfenbFdsBG9pkV2I+IYHueqTtuy5D/yDPDSijrlNCPrExo32yhUzXYUkTd6k9LKGu67iBh3UYH3QoI0nxRwl5PozYNGBJsXWSDT0YdVwHhjWsX1m+23OMpO7arl8Kl26h0V+aDQSwHzVYp4RMyoDMYcRhW8jDhkEdK2kTk5WQonc4E4ckO4RaHRhJSzm+jcMwBZkny0a/JPs0oBq1KQPJcY+5E9okdZhPnYDqyYDeknB2JO7ddipUz/ZEPUm/5wBPH8YC3aceQ/x8O2vY82yqY66xlKszwk/vE9xd243HEzu7oFDgFzBkENMJ0A5iHRpIFmEnoO1D1yDxhNRxAyaJeTgaYajtEYcfWdX5rnzRn39HzTSwY9gEpp2ebW+FHCQQs+W6DozGB65/lUDTMrhnMMSnZoYZ1fmgR3zT9bvY0YnkuIN4QtgXGMQXZlAyXLSNMkoC9qEmis9JLdnEsVgH7ewgmZMsFd0nrOc7oHdE7IDcC9bpOVyy+a2DDjUZl08Q8F/u4PsBDnHwKsdwB5JPrnskYKpDu8JcNR93STzUSYLvsbXAyOsFnVCNWxFnHLPYCY+x5JSPl4FLERqSuZZlh1HZFpED3urxvfgIsdkGIn1w8JEluXTib+mKDLnSY36MCzQilnQbB8FHCM7RvvHYZM92F9wwNmZ+lwB9gvsLH9/dTkUFgEH7SGy4E5tNHzHEsAYnJTc7MTkGoeuzRRLXARFBGQ5MLQuGaA3E+Zrx2WMQjGEaE3mkjTXNJ33hPyKtvM9vKliW4VQjAt3tdmHhFAUWX/zwcyS45LgsXuq4feInIuhnyduDDnHa5MYDkxNDLJVM1zawBidw4GilDg7iuzbWiL07O9MOzxmy0ztEvyJGIoH+k0APcwUNglsaUxQyfJnmPUXAk5s1vUUmnUvnOAFol/g+MdoeZF3SAVmIP01Y4N+jpC/UW0KUYchyfPGM6A9cF6DLqBMCtUEDYDocXwvzhGgXCQ2VShqBjEOmpNLF11Ykv/BqSKb53TC6QsDmE17UEReGZrv8jnmQp7D0ImcD+1cBwRZ4sbO4+pksPZGBL41eNKafkVDccSW09peS0/S11xRvdtGDQuZfQEhuR8645wfcgJ4LNz3xI/jS4PnYilAQjJIiNwUP+fjzcL0/HiM3bJGLRIO2Sf2AtV2nzd0/IrSWhZKk5Hg0RZoK/bD4ocXnReSn6mLfAmwWiiEC+u4HuUE+9IZajzEARVEJ6H46ymljKDZHBYoEwBtN/BdJZ43YHF5eO8DgFRTbqDXsaq4doKMe4+c1UCVcCZ/eEAIjOSDaFWDfMPN24d7rQA4HZOowWE5xQIwJYn5PZP69tejm4WoBaWWpSLrzpwzjI+IU0enufqU0wEFbh3sAFDtZj002c3uM8/QynnNrZllOqx71sR9PJg3McBI7YFgBmhLTw5wJR4I+dsbeLLZF6QAR0BjA/KTbY087ykQCMAwlxrtZSQRL9I52Pddn2FnYW/PdKwADfOQ+GX1bu1PLpvQ5NvOdxHEVCGa0dUDlhk+ckaizdQ/sOUv4bFhkEjqji3QmVHVs63EozqAySHs3AkxKAcN8/UTenyaJBlAI8m4oieEyODuXYlp51z1sO4C121ClmvQGNpkJk00RJlBYYI6LBth3IPDa48Q+toVpYj2diSD0AJn/M6lU/VFhyrPXSEmjoaQsibR6Xw6XRDGL/aTlY4OC2eLpTM4g1gZywaMtgmSQLq93NkIPB0TN84UYQiMtL4iykFifTfEzJh8f5k4S2BeUZGMPUtS3Yd7Tc/jDHEJwJ3jMQPAHaJ5DjigWo4POErQBh9+XblFqeZj8eWy2geYaPNJs2YceKfC8liemaMM6EIH98LIXPe8jxZ8x6nJDPobro1T0GF0+htm3DvUcHS7edEDvVN/HQ8n03W7ccPUer4YlnlwDqY/tHgniiYQUuF0SFymXtyv4pxTCI96qeCRAiq2hBEpMmkNBhxDGO0hkgCqtVoufpsXHeD9ITEo+ERVua+jo8f/9dwTKdDJO/k8HaNMlscM7YvZobDBqRmZ5syHw9RLq+Xaco4USn08NXNNUtjTAI/nshiEX6weWWlbFV/NYVV3xW/lkAD8bNVWtqsu+yl1Vta7cD0azWq4Mvqrq6dfKvnrQLFfUmnXTbHzssKB8QImVqe2dKx+b+a/9ltejnw5yp+n98+bJl4P+2cEt+zSs1SrrZ9bVKS3vyR26d9zbrxr1S7mhpcx+0/CuP+Q712eUHvcOnHqnYX5m6ud8+dDPqrWmc1XN6597PWf9JHetB1eDvlmzU9c3VtXdtLT9QX0z3VBTjnqS++j7++mTdetWPjFkdd9MW4eFyqB+qViyO+ydFArdajo/aJwXjyzLI6dXwyxparc5XfOP6gyr1nHzcLCHg2Fw3Gs2z8+qtYH66dhrfjU+p1LrVuG0cJ5hsvnh07Xaz8GeH9XDgnowULvW7UlrvXfRItXzG8XM67eH2ZPGMNcrqx9uy5dezcvQxnGlKl/0PmVbBccsf6w2agddla5v9qtKx0l3Cuval8H55aDh9/fqnyvOpVmtWmz9SL+w7UKuWNkflDc7xezBQb2VqV+oVreZuywfF9lpnTSK1XK5Wc/sWdmT1Fd9qKl1sOmXDyn1uI5VclCx1cZt9ci6YFa+/Mk6Omrula/ocY7UyueVck2nstfxXc8B3/Auqnvp2/RVy6yYrDP84DQMXAsapnzYrVcP82VDvf7yxcMsaF10DQPTomLeFrOf6eV13uv6+SP3a6VF/Xq3v1/PtM5amVpV0cvH5ul6w3a9erYWDHLYus5v0gvSOrS9M6fcaBLjwCe9s+t6pZs+q/lXrdZNTsmfnQUDFSRKINENZvE14dZrHIf8D35Moh8brgcY7D4kRQ9bkqQlFBthzP4Ney3vCnZEU1XA5BDBw97gHo6O4iGQnm15Qwieujx8gWwEtPlYAOmBb8ErBw638QBThhzcpxZmri/Bzp7mYt+QBj5l5BSK7Pj9XnDY0V73fVUAf/HYVFnBO6rA5ZR2CdQf8XHLfWGdT7pQIywsvdtAiizLAuNB8gW4FxcFcjTfqdohdi8cbw2MMxhvQsfQe1TD1IbExlzEid+JzAa4zgF8DNmYgs4INnjpsz6tu1F3+Bt9YV5GjRvDsz22eUQa290O7/Vt6ni90U0TEze55t7EIjcZXfowGV74IIRYPMt39qaN7f5hMy42UCynm5mcqxZiu6aNbyTHueE1DSM21aTxnuFHiw/H/9lOjU41vdlC3RZbNj83Od3MhcO8v1EKW2HFkA7/GAlQQg9r/1mVHXnGtfsydg25ToW78c7aE+NYdPYTa2hSY+/E/lwbnXXt7xgS9+5ObKoIL6E/rnsu25oiCwe20ELTAK554cmQKTvw+0jNu7Nm/Ll20ygOXsRorxIZfQwQCiQJ5kLjy2g8DJFX8p4/17jyH+1FY/JX9CZh75d0jyXPWbEIB7mfe48e9KLIuvKHPeoT9nF3MhSRcZcJ/wxRHb4yhgEkJlAY1RGCC1HD2T+sJeEWLewLBobEHeiJ0SHWPGOIzHhgbBcBWtkDl+B1GGaiT4byuU3kmij1OSB+kKL94S1oz0rJAznFtZ/SemC0VLtNDMqE37clPp6UpaIkJ71hJsnLX5ByGNKDtVL31vKGc9It+dhYNOKkqwCyryy9XInLQk60DXYiXGP06h3bPQQjPkLDG7+wFcQhn2oGsehZ7RB+JJYDhanM9cad/2Xv/8e7/Dex4xXxHWKv0OMrocdQ/U/wn/GCFYJcIch/O4Ic+/rTkMXMqhWKfDMX6etb+4eQZPT9RI2d2OTfIjPEMApFA/N/Hc0auoJ1nC8YWlExc5uFHM7OhXTUv1PGdi8hksXLGDJtF7OMEs9uoGzi22dYfL2LFLJgKppB0nIxJ5NsDmeKuiwr+fRmXjNyaTOHY7s/O/VH5c1oXfA094A+FrHlt1Tz/fB+KTZ52dh5GvqfXfZzc+UXGsB1Sm/FjqhDDYgoRB10/5bMH5m/y39/M0U/Y7315orc18LLUR93b/WufZknj2csUkORlN/oYUp5nJWU1cPUqrUQ6T1PaUwpq4epVVvht2grKN/xXKGsHqbeIMz5BSy9epia0eijKyZl9TD1Vu//1cPUvwU9PqXQVlYPUysE+TshyCc/VSirh6m3e5G+vrV/9sNUflMvZIpYkfWinMVmRlM2TSWjb6Y1Q1Z0Tf8lHqZMnSgkk9ELORlnjdxmMVc0ND2XLmh5YuaKyuph6td6mFK+771EWT1MvVVF/6YPU6+Kl5c9TL0xJb7Mg0ekwp7TPSeUBu3v/h+335xC</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_16a3675ef842438cb85db5d1411974db\")) .filter((elt) => !elt.dataset['step0']) )[0]; root.dataset['step0'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>\n",
    "\n",
    "\n",
    "\n",
    "<div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtfet62ziy4H8/BduTbkqxJJO6y7I9Xy59yekknUm6p0+vjz+HIiGbCUWqScqR26P/O++x+wD7Cvso8yRbBYAkAIKU7CRn9vs66UtEoKpQKBSAqsLt2POvjSS9CcjJvucny8C5OTLCKCT7hu+d7M+j+MIjcxLHxLsYzbszj9jWZGCR/sDpTVzL6g7t8XDmDez5wNk/PU6WTgj/R3qnHSeOnZtr/48LNwpTxw9JbNwaH678lLQBziVYULxwgqmxMXTAHT+cR4Ayh5T23Fn4AfC2iMKIYk8NNwqi+Mj4i0P/TI2FE1/6YXsWpWm0ODKsTndAFlO5xGVM6ovzw+UqPUtvliCQ2Akvyf45sHBN4tR3naDtBP5lCFz4nhcApbkfpAR4uARqCeSTht00IijKT28aVmfQvHNhR1fRNRVUmfTd6IWrxYzEQDCM0sbRPHJXSRPIzqLYI3E7djx/lRwZveX640iy35RpJJ+1yYj+mfLijgx7uTaSKPC9Iqum1E4CkCROVH2paz3KQuovAUdS5KmxjBI/9SNoNmcGPKxSSJs57vvLOFqFXpuzTAvSMTwLABaoOJ7nh5dMr9wrJOuH0EJtck3CNMkK++B76dURtF7aRuYga2ogZ/Mg+nBkXPuJP0PFKVfrj7YfemQNJVuWVV/LWbTesZbRup1cOR4WbdF/sFq0Qi2e0IUEXnV9hXK+JjVsuYHvvvec1LlLiwWRgxK9WJAkcS6JoD1Zj94cH7Kx5DiNCUncaEna8SpsX5EY0hI39pepQXXTdJZL4MFBCRxGbkrSdgI4zsI83cM/UGySGhkXxonRaDSNk1Pjds8w4N/5KnQR1fBIQmIfevkf5BeQxriBnQYADCMm6SoODZr6COl05nG0aDhpNAOgltFYUIKLjht55BWK8lHasJrNKWBv9qqL+Q7EkPa6RUGM1dlNShLg817lZUTmSBuphOSDwQuitBqUfGe2msPYzlF4BRnONq6fhf8tPNNi7sQxYzkgqfEElWnhLF9///gpaOZUrc0lSZ+AMvrhKlolFLhx7QQr0mJqCJiIltUQKc6chFzQ3tAyovk8ISnjw58bDNU4PjGsDMMQ4KE61pSnMswiZWOQICECkdMTw64gInLWCUh4mV4ZbaNbIm13ZOIZMSZiN01yiqzIh0ZDT9puTnV8vHDSqw7IHWSWE2uWuCjK+dqwOT9CS8dKhc6KIs7PrHNkygYWGLmmccDJG1VIxoFhc0SxdVhhl3WF2fctzNYXNqsrrHvfwrpqYVz/z+KWcdkyZuf6TnsTgunkPordxA+vXhOg3uDlvSc3dMz/O1f7wF++cEC3Y+fDCz9kf+M3J/G9s8zUMqeepA7MZ2/QPvF4EQ3ASJ1VocGo2V/5yXd+CBNDg2b94x9MhWCqaqybxiEiGMeGTdr9Ai+v4DrTLEWbcwBKKwHLDIk9pMQe5jD4hwIE0WWjXOoBx/49TqFR+Ncy+tBYM4CW0W02c93eCFqc931JjsaJNAZgPpOnkqGIHzWGyb+ysio8I6vjTIXMay1lZKMobeUMauGsG1m7c4aa04qaHucQOZf/pr9Yt4PWwj5jnOwpLU/Wy0auArIMoBsW6g26k4NlVcubPi8DRJWVdMg6siyWQ0VwbPIAwcJYC7i6LpN3PewdU0F1MixFdXJi0GNsS1aBfHjFrr4kOMqitsO8W9dZDznNjBbvWRkR7Wzy0a1VKH0myeJTp6oqIZSrR8Jo4YdgY8S5DvthQ1ABXbWVoY+LgLIgjHatLVSyjoLoUrNJTCltJzOsbUAu/J2aTSBXTLGlWSCN3qAh/SaNweJmc71su+WTSDZ1SVPM2/hy1nhwG2+MB7eX+L/ZpvlWO92gfR87CZhVl/crUYBAdyQEr+YGIDpduzuE/hnDEN0Z2YMu/L7E39aoi79nxSBVoJ0adndcyJ5XxqQOkKlV6QyEOjGmJFClouCfB6+cFBymEGwgaA/496YFbiJNyidKaOQGqqlPDT746zgD4UYWpB0cNBULLY4+ADwHPPPPMwXJyb1j5N4BOYDNSb0TSfHJJ/pw9u5cTIVC0nUH+X9N3LSB1sU74B3+8luG3RJsvkIjNyXVYox6/qWfUuP5VewvnBib6ozCmn+Z0z9mC37a89Fo1qc/5/PR3CL0Z9d1rK5Lf3rD7qg7pj8n/eFo5tGfY3cw7M/MFidIeqOR26U5M3fmddlPezQj7twEGComla83BFI8mbPRHP+h2A5xR2TMOZvNRpyHsTcfOzx1Mp4M6U93MLO8AfvZn7iTfs7ZfDQbeowdb+bNxoz9CfEcMsg528u5c0kQvAEvClgaTVmG4rSAZzL3L0s+iwcjzk8heQL42QhHdQ8ambotLYO7MH4CI5vvFb4MI9jKB/RMIbhxRqEFLeQKQrswcGlG7wP3qjEYfI1Bg6Y53dMoEhQFHXFEuWE/8N/mtJ7myFJoljoWp4tGcE6bfZzlenpmtYzi3/OWlGHTVLuc8UkwzpslP04WegcjFDANQJXd3OE01S7vcrNfsl5KpDIoabbxk5fOS+Y8NsWOXpK4OPDdtfk05HBS6A4GVB7wd1OgfK9GBBHbgojtc3H+rWyWvGHsUlPWY1WVdd5UTPDCNYc2fAbtAI7MjSRo1oLYVCAXTWAB/PGWUWq7skh10/Snaq7amfnjW46J2L5789wvS1tWfcu179V01p+i6QqJ6hqhqvvU98f6rmpv6XRqE/GJy0PHl89hfMibyvZOBneKUTDF7rlvP71rc9+1we/d5Pdu9NoOti3TFvqffSfM6sxz0eosem/enMeGdefmtP58zWnVyd26d3PemaymOcWmK8yWrIGbctvmPv7dbQ9tgSUSUh7+kVVAo00Zp4JKSU23ube+KF7WNkPSMJcOOAspjw6bpfDbR1mUcq/Kss/oWHsump0cBPhZARswtRLvi/X56a1PdSKsiJvoZJSHbVo0btMSAjd3b4KdlZP63rOoUi/RiX3lxGny+OYpguaOOZWL6HCh/Ebn5RRotm7L6OlzQLCDrRBDDtHHv3ugQFshBxySYvTvg9HDvwcfhwl/D1FfCu9TjDH74TX68SDQuQOtJHZWZgV/bdjGV0o8srCecuw0XpEtWhiSSyf1r0m+hHhcrHBmMAvnEsztlSctQag2mxCvwVXfHKeD4zEdipudZBn4acM0FVOPIWWLlcclxeI5OqMBQS+WCAtlqnhnEuFzcYxHKS9jukPiIiZL4qTJRTTH1epVEEjzuCbwJ5GdGgcHvjrn8R6epMBNy0h8j3AeOJeMZSEgWJIhAL6k23S4dAC2KQPnkqNDd7kyKk/lYUUTXGMiU9gydhNVxXxdVy4PNt6lVAqqFKu1D+hYWLIOihmEN04xi5RaS/xdaSEoxlDWn5r1Vkt5Fi2ZSlWTnzCunynVO9PbfoWTp83QGITnFdXVNLBYf+3AI5pFYrCLAX+xND6lpaHdK7KzGHYXb8VSCpoRbCnuUeg9Cz3fJUlDjWT7LB1/JKAnFFXZgJQN6Gd8SKCDcLaWBEgGjAQ67EyzIOfMRDvGPKeWDMw6JHYCU1Q2WkZnuUquMgTKqKmNPZVJqhZ7xnq+aYZX84yjOms/Mc/P5YkvAz4xOFTy3l9e0HHIVJZ6BHbfPrjVgG+O5GQSepD4Vu+N84KP71ouw9NR3YZjtHVl4c4kXK+q53uX9mCNpy6alRZEyuxRxDvUyGwsnOQ98YxolTbNe7F5EUTR+9WyxG22fmN8843xFcf1L8MoRgeRjpY1rVPNV7k6TFWT1SxJwUajfTdXQcbbBV2rNs8VdzHjVEat8hxLHK7C92H0IZTYq7AaBDyxsKp5aRfZYxfUiR6tO3237SCKvs+ebu86Oc379ICyXt2p2ViRCuu7ttrWNqvvIRXttalyRMTijv96aipORRSQDonjKG6YvzBexLHf5POIdmcX3wXACngX+WHme0gbTB9BI79ZEre0SnsBxp9z80uY+sHf2Ybvhkcw9ke3J7cMh4JlshPmL7Y93E9vfpolJL6me3f4NlgSJ4SiZVmNBpi4sU+SfB9z1lw8/cw67/gC4mssHfTPKk8+fCf2Cyd+j7vuTwyB3c7vKxLfvAFz1k2j+FEQNEx157YoeVa3hrgqkXlCJMD+ohQmaxCAdGKyiK5Jo6nT5JKAOp6fQB1CtDzUpmwZt5t8pzDUIkkfheA2IH/fxc6CCFvA9bQj9kNsvMyO0W/rnq38wHvEN5l/51+uYqXlXRoqyeq8TU9k/i52pS6zKOrlHfnLLKd5xCLYGOXBL3GrK91S/9hJyLBfAAmJJdinLE4kgdI0EZJOWi9gXlIpKxkiDh4/wIiBR/egc3ghUYRdZ31XAC3SRMgbDeSNFjIJYALwNOBKhogjh9AKFFfZBiKYuP4ahsFXJMY9IAWClKxIckW+owY2Lj48E6xfSapVQNO9YpDCTkzbq9hrJTYpTJVzdsZAmCYpABtR1BMPgpaUt1AohP1wK1l2JKGSaIaZXqFjgqPqt2xyWIXJarmM4hRsII9O+83yXnWqdxdoKcmFskMiilY2RaHlkosC9HD5PnhMcFdxDGO1nJiQJfVgLNGFUeJI+UbdQmU7WeThRk1qKhvMsAA2afLyi8B/zg+yepB959v6Kf9S4katp1zjNEqd4EkUJEq9o+BXPCVF62mfFxmsOiDVkg+nCKBUb+0mugQAAKcAFuJmPIiJ23EhpwOmExhU9Ce1rzKwgiMmssIuEetW/H4IJA+oiKEs3OYfSmco8pozajmepHC55F5HHxTJgeb+QPzLq7QkuptdRXdzF9HdfITobupFxytX/N4iuqLqguwQsVmhinzKcTGo9gZH4J+oaYuF3m6q5aMM1jsIScEQJFUu/IzKKQTj41w4ALSncu064LYl4rpdw4vc1QI6XseNiZOSbwOCXw2TgZr5GSr62aGnEHHjd6GaB0YXz0ZkWw8l8Csq2RyetoceXhDqE4qLi6RknUq8cqp8CRVyG2bXoyxqBgm+3ZoeTPmR78qWgsRFwB6MT5zesAll00hYAslmztfF7ncx3ly3bqXdnKdF4TzLtivf5A8FluAXwlkAYWd/aWs439qvbgsvLS6zDTH0+MejWaIrMc+UdyXkiM66BpFmlhjWN9QJk295IaPUDHw3O91vjALB/fa4c1sVjn7xnhuQdPf7i4xkzdmm4o+O7xaW2ipE2CqE0jKsjj1oTnetjsQTJh7g6b3D4lDe1kg4XyzDE0Ia1fFDTRPeRc/YyaOFeEpnW6WszmCHFqlp4TZWh7Ywcsq+tkRg9jRHO8B4f6YenxRiunQABo9SGYGlwxY5iYMT0UMQB+KH9QP1VOWQuxDFfCKYI6iBbGyylA0VNwX8TT08H8aKqd/AM9UBgdE5xhNLt8LiCbcKGGyeWgku/RQcBMCnhvNZLq/zstJlUcYTyRrmA4DxV7p4YxwZX31VZFfQ02xjF+MHytxyh+3te6VFO0lHJR3U/2TGlHfthC55Eq3CVNS9+xpV3CLKdAvsmwOhcXMDB1MLO4eCZQaRDFup1ZLBJqtvMSKIfJyeCIYarhWXFmGVT1k25WrI1VRkJ/Pe3so7dE2JO5UWygyoNCqE1qzHr1h1Vj5nYG+9r1433ijDraI7J7k4tJsRlNJKZYnzhrFX3rIhjQPrarW9jxNF/Z8alcUh6uCk8Gkq9bVSW9f12oriXMu6uq7TVeljvUVP13VaWqmja62Orqt1DIWEGqqXUrMOWaueO6uLZL6uVaVc1ynlXmUJVZO1/JdmCO8Efkh+5U6JPa0BTNI4ek8qVvKrKD9xlgic/L5yYrIV+j8iamqZC1zdNT/rFLtXO7Hxyuo2nLBV9PycWBtNspZm9NsGk+c+FHXQPjf++lc0U3FjQg2GMK5WoVRNqbp51K6cR+0v8+iXebRmHj39dPPo3m6Tp10xedpfJs8/9eR5+gkmT/p/MRhWXKdCUilO0QjJh+y3vFlJyMAJXRfkaPJ17IyFqkhbQemOUTLcb7I9MiZG7kprtGpEMJcE25fwCpe18LKDm6ozDtRPFq4josEHGoDJJjK84sOeao7Xi1g3FViyQHCTM95dg/j8J6aeih25iPEXlyidA+SePNyeSlOUEN0WsDRX4YixzHorLDs1l6yCVIh5f7LISrY4zIgU8X/ooDuFUwzOmxwSp/SmdwzfSEgbOYqC2o75/LKp0kUKvhy/KaSP4/wpc5ba7XLF6xaXchjKrLF9mOdBvchbBatEhqdXP+U49EvEU+tXfH6d0ZuWzrpGsafc4SUiHnK+S3v0i0vIkMCBZs0n6yiYz/sHQzotFovk4VKj1uL4+/lVBHqeKEL41KrIzf1V5GYnFdlqr6o6IiLUKkm5hvdTEgnxz6Mk2SVrany0ZWjDnC3OThG0PK9doCzuw8R4vnBJULEBC0OXofcEHE6vchnQ86/NpnyRoh9SqsK6nLqiwgq+O31KWUJjK37ZAl9Bmt7ema9GmsKtqOZUC5qvRO4Ai/cCf0evBabefHYxcAV0GjthgjvNf4r9SxYASKMljAHzKvowcb2KoyWJ05uG6S+cS9KOCWq/H17i/S50zw0IyTObOxBot7PbR9t/RNECCdg7IuJVY216C3IClgnF7C/XZiZu1hyyqN+6TuA2rp24oZSLVvODW3GZeLNcZ0cCRUp5S+xGioFX0MrupqWnCFBmYB6YmbIw/J0bSQSX5EQvSUbh6Fg1y4vPqtD5Bb/Poay81g9uSwP/ht7e2BmQBY6zUN+swhX0fo6WArn1TuRKuxiC4LkzI4G4uYOvKtH0V8q5Bz634Pr7AvoFhaEhNbp0HuCntH5OUzINwpuv3mBHQukvhXiaCMWuSX5Mr9pGOKtjQw2EC5O1WKjD7N4jYSgyqWjwzjabLss2rM4QB9dyIzZR41iG3CWa2VmUzeffM0Troxyprh81dZIQe0R2X7NWZIGojtWdkHstQv9T24u3FCNkW9bXoG8Pbn3UP6p+ejw+ngi13caJaroWEUoNcxgFfcIG+cyJoBk6VhD2EV6/jjITRgRDq+hcy3OPtjRlyfl5D2O7i8TMzWffS/XfrFLSqBlHKZDHYbM9sTxyaWppa8ZlrlExDvvaYmJp/thZ43Sa2jbur/l31uCyYV2rwhrllGWyg3buyRY0qOrze6tEgb61dwmgO6hQCXpWjP7VQJkW1MHA2J+tDZnY3tWQkurmStmOqFaiEktaqaiy2PsLyvcdIjQUhMmYwdxSoCOhHi08ZoVJlPamWbUDkAZeHkdrklRb8B/lIxQFdNzASZLnfpJ2wGIBSzecRyhK/gZDbjndJTzEwdgVh7XbRLkXxHYVv+FMKYqvrCQIrO9SbXyTxBSucFbK4jQab/8rfHBb9JHN2VtlAw++flFirapQ+myG0CsZMo+iGiZ7ScNUcrOtOGUpcQC2R6w621nL4rYVgCQlS3FtQy8KJk6Gsk1q5rlhKnJienNPOTHkXE70URRTyawWEweoElOWXSkmDqCKKU8ue1hor/Yt9wqvVkn4BFHMEHXyZUTLsnMxcAo5T2C8wVdK6FsV5DqVDPaSLACgA1W9JClLKkIfim5VA+7JK9XV25b5/RIqJal7C3Ghivi/2uQw9nyLtzbgQERguMx0o6UTitqv7oNcHnQxihMAhU845HKKxfiTpXToRjoztz8W7DicWTq4jvd8w1CLp/jxRpvOgB6ot/EGGnGFQK7BT0s+jUv852XvwnpESYgDp0w816j80nDxvAyaU+WBYMdhoEDWDgRCNuvppq3JoZ3c7GqyePc2nfCmnJnVSnOgSiRB0kdpGvszmMwbJm3hlti0SiBuHuEzW58uwMcpamduBaTeqc+gstH8f0T4clZbJcKzCwHo880D6s/TY8nzqFmOY+RvNFV2sLIRtpNAMsJMIi9hoMJ6ZqnmVOVBmcg/HQ84GJV5wMuwFKC7hevK7odCaOdoXZkSt8DyF5w+YRw6o1llYuYAcnhOuGN9/YI+6gYThzLSYQAIN07RWD+2F8xDj/E9Lz+8fBL4wMxr6UywsOQTglv0OhPXg9uMEndV2jlpGnsBQb3VrghlMQDB5q/YCZKbuFmYTsCRl3g4SIf6CIoDlXOtgRdjrXoXCaUtHtPGWlA7l8tQIbVFmGwJJ8fPtO+0EB1LUW7e2sauYufL3LNARYMGKZoGzcLge6NtdQZof9mdLlkoN0R8TA0NWU94/LakLIUQuLbgwSn3SrxkT7MX5d6kLTlwou4qkfR8vgqyyme06zS58G+36TEnmiPgdp6t8hS4kU9NzdOj4tmODIpWuZUVRr+k1zjSaKnDguQCCT4kHCrXo+L0bI5FMwo8+ilhZk9KllFZToHLvgXkjeZxlrplB0EPaOi3bUgiEYejugWHUriLvYZJI3B5Abi0ItCHTym+VUFfGDplCYpDZ8xXg7YRy1cQRGo8iCOQYykFvY1yKh0nJqCG5gvdMpQU+1qY1q6WMM0QzP8ujhZvuGWqP0Yo7Q7Aq068V2t2eQZfD6epjQ9+6EUfOh65Bg+DlkqBpJv5K2DwqStpc49ajG0cikXh57bi5Md/ljnAI+/dKkkXLAaoFFRJtfq5HTdJXq1/pm4ffaEkTth5/oZqQ0tyELGUussEbbU6iC2hP1Qq0TS+Ft7TOCldb/JxS7r6+y/vRNNZpZFZ0Ux0zRL1UKyhuAntobYlqzrVlgVn6GO0QOHWKtHCkpbQGH8fnNS9EvRW7TF8UwrxfPmWYa5WC0R/gbmNt42Y0IAyfaL0wW2F4m28JQ5BSlgk4G59qXyDlc3vjFFCAO4VdRpbObowG+nGA3G2UiouxC2yH6zgcuChqtRNyYmsDlrIlawZ3PLb4wKPrTML5NlVTpCcgdbXeU9qucSNoyB4LNlkt3TS1ZWAbxlSDlrGjFw51z6+4mri/UpOmJobtQzRkablPAvT6O8++dC41aADzSBy30NKSBxQooLgRj70XpbnIlolVDNQpmoMLbv6CPeZguTEDacY2mKV+8+WUXz8Jo1pGabmBHTujrJuyV/qpYHfKCTC9ZGKF1cFWHUTdH4a8uJauSMx33yU12yXQqmgq9lja9GSlb7F3aSbZumB6Io1tFIZqWK9bC+CHvjYXgQ2h7wgpr92Uiq7WsA1V+Y0pzsogipqITgBphFADNU8tJiIly2U5ZSZarLkg5yAHvmHLEqgYPN0HTq9UKzoDtDTRT5ODcv45htJZCLwgQLMXC+BY9lBVKTFtz6Y0mW3Cgz3jhXPcbelY7laWdUV3dE4b1o2tRzsVPRBVdFVlZUksqloqd/ylvohc85rmuq3vKlyaLGtftB48wpvtN/Wt1W2XvyRjfXbxzRWeXi5Q1v9doe2KhbHzcojIDtNXxGzCNTZa8cpZqcJZisjNE63bQ59nR01uOdMmuGXp9PyqRgEuP/keFHMiqzQqXqd6sV1+frXei70N3vkAVB55jGfYAbxjjBYrmjjlsuQ1cqUrjvR/dltsjLk57L0R4rkE1XbZFJxEKuEpj4KoL8hZCdd/XBFSKDT1Wx8dIIUyi0HtjwSpM5v6GIw3GaHpRRcITa/W/4pmTugOGpItHiIumS0N/HhhSKf0VYevam19EWZlAFxX7vCLncNYNZAuOfAd7Cjy57fTyciSoSm6kHoMkenRn9b7donRl/VP6nU4wp28Tab0nU2MrvS54F8glF31K+yHsdGe2tFDrZV5LSqIn54p4q0t1dEs5gpktjdBy7/uN/SS9mpvFUjubRjuhT3NzHuhyFc41DR4YfKmNdQJCRBS3FcFmwWCvtPdbnnU5am82WL9tLcVrt0ksS/Jkfs9ZaNtCam20VR14DaAEbpwloWqPopJE/5mz2f7rpa+XYh8aJRzeZXFrzgQFmKBi4PF4ugLHEqX89UvkJv90v0StfoyZfgla/NK+UzFlJ1t1j1xXjle5HoaxLs0QY++eNdE6AluCXiSHhlCTSHz7rKydj8YNE9dpFWHuGxXWdpVkEVp3dqwRY04sja3LQ6owFZVMIKFqsf4u0dbdlzLq0r6I+a6IHROJ4H9DCwya5/roatP22kkVjlOQoVEmuVe+ZmtzMwd4rvbgtfV9V5lWKBtJGWa+XIxl7d2bD8kJfUCWr1QwSpUg4RpmID8p6y6PwzTgulS8wx59ug5IvvqPFVtc7IisYRT1Lr3hX3LpfAtIdfDAzKYxcAVzNfzrTpPi+YibLKZjcNbMjibSX92s3bJegsuNIe1PFceLG2XQcnbX93SSherFMuWtJ3Wwcou0vqhLBR9YHNAH8elWD1/bdpBY/1/NtVIpv4Za2o2rKUt2X5Dn9+xveWvxWg2ERH5SRmx2mu0z/SJSL0Zrq3aVIrLL3yqQnwOorSl5FHGs3OVZSk4G3Ow6STBaCyGxnh5/T4ECxnf5meHh+mMSGJC1NAO16F7SsSk9Nj3OBu0B1WJ/vzKPDwKY+LECjvnx5TOZ0e04UlAy2Hk333irjvoQ77WpyLNLq8DBD1kCLJ5OltHxfObAZe8H5d1jdBOn3nrDtUEAa/Fr7Rb6FP+K9//s+21bHGxv/9P1ZnaJz965//G74nkPmvf/4vu2OfG2EU/kHi6MgeQtWhjNP8L14fsWBw+MMLsoYUj3gKU3iiG1TGu3BRBaAZa/MhE/dp8rxM/Be54ijIyisY+6evs7mYNV2n08n41jYbbVPeKKCoge9SpT2M3JSk7QRwnMX+af6yFVN8qj3sC5Vnmgcv5vRC/1rlmjKwDvDwJorC7P0LTiJd8vAAjovSox8NMyWLJRoWSIfEMQgMTDg8GlIYm+z9gv9489PLDnV1G0iww09wq/RY3c2m1KuBGqLk738Yco/o6F6vYOwWT19gr97SXbK6ZM0oVWm/pl3eJRHoz+0+7l7cPzL2n7IAWX4FDtj3GNYDj9Zw6FJ9s2P8gCbmIUvHTVfs3QIaEtxvGfvCawVI8fsfJy+vR/ZNHPftbvjk8LeX375/9WNMrn4e/nLwbHw4+LH//FX35s31aBAv+i8Ol48+rK77y+X3z0fv/tZ9P1mk1/51MHa+Hx2Ef3vZP3i0fP7q+vLkJC+KPqiAJfEeiRnK4wWY++hv3z4u/8foCLcnAOjtvmPDXzYWYMGPPngj+8XpWUg5u93HgwNIFkAhl3Zh+glQhg04mEpPDUAquDv70I0oqXP4fVNFyyrTsrbQUo4rIUFMlq8wopXKxFRcdoR0YSKGxDYMUd3+aNQbWN1efzIe9AaY56xRDrosHkiFbObB7efXTiOPHcu2rMFkOO4P7P5kMulNOAS9jbqKZvYsM9bhbDJkBwJ69L1Yu9dnn32bfg5HkDPE/+jnGH6O8OVb9jkZA7CFT/SxZ3Ut/EYebJu9omtjwgjfgRywhC4kdK0eJEzYe7V9PJqAJXR7DMLq4uEE+j9WyBDRJ/hsoGXRBBuPMIwsmooJowGyjCWNKUYPf3Z7+TcSwwmCg2OJQ6z0kL6I2wNqfaRo2+fnqIDS5n0Q0YipeVUMnGlW1ub80b/9DIeqF3aefYzR1IKdySD0PhHa90DlCqUtnvFCzcwSMkXdVgb83lKI/fGFnG+v63+FBoz3MhzL3JxvxCGYD7fwEybXU3VCZ3/taGMoU/0/jKd0+9CR8eTVL4bFiX1zmU71pTAGcIqnVicM/Sy4cIQGB9k3fA+NofginxF6xPNGE8/pOpbV99yu4zrDkTebdOeD8Wjg9Dl7n8kEMygY8e5pirnRAp2aizr7iOYBeOAsE8xEs03XJgqNvxgc6JOYeLUMVTfmZzYFabUaZ2fA99jq9QfdCYwi8NHrWyN7Yvfox6RvjbuTodEyhIG8eGL1DFK7Q7s/7PdhACtGcgY/6A+GvVGPkx317WHfMERku2N1x/bYQngAGY4m47HVZx8wWo77ls1LHgwn/clAeNwV2e73e9bYHo4ZsgUqa+O0APB9qMR4ZDBKfWvQt60xvpXOHSiqpLxBmxXS/7iONXdJl/R67mhgOX1vMJ4MJt7MHdij2ZDMB5Pu5+1Yn9+36dlU8wfjTPXHgy/ezRfv5s/t3bz8OTm4Hr/74dfnh5P1j48Oxj/++PN1fJMexv3J73/8eDhKHz27dp+ur/44fPrC/v1wsLb//ioEjygehqOf+4evb66fvEp+/eHlu9Hhu6vkMI2+v371xbv55N6NNexawwFMVQOw0+3xWPBuNFnbvJsezDXd3ghmn+HE6g3QOVHcGw3RL+7NF/fmi3vzud2b4dgd9SZO13InVt+Z92bd8bzbc8f2zLO67sz94t58Givv/3sHZwSewsSaMH9g3EdHo888i57dt6wuz+mOx71Bd2AobsYEXJdJt8/cjMlo0h1wH4WP/WPq+vBRXkEeD2BU7fZYYTA62106AVCHCFyXgcFyRv3B2O6NDdm16kFZvWG/R0Hs0bibUxpao+HAGvE69Luj8RDY/lgH56PtXHZdyjOP3tVYmGajeXfmEduaDCzSHzi9iQsyH8LcMPMG9nzgUMNk55iECLzVzxKBtw4H51NujkduDBa41iDPbGaUFQfi4IqVvN/JxXmB2fsl1CdBRPcpZN8dFxNoMfTZlWmRExMY6Fzyq59eNST0nOg8di75IUhl8fcp//yOQ2AdMmhpATEjjPma89N54yrnp9mDrtkBx5wyfPJl58c3z4B4ho0vG0zZU8DRKnbJU3pzSIUM/4Lj+b5xYCjo/F4eUSw5tc7cj5OsbFozQChyCwdlI7eEVsjb/JH/B2mC0aw=</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_16a3675ef842438cb85db5d1411974db\")) .filter((elt) => !elt.dataset['step1']) )[0]; root.dataset['step1'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>\n",
    "\n",
    "\n",
    "## Save and restore as pure dictionaries\n",
    "\n",
    "When interacting with checkpoint libraries (like Orbax), you may prefer to work with Python built-in container types. In this case, you can use the [`nnx.State.to_pure_dict`](https://github.com/google/flax/blob/764e1732dcd3b8bf178b9ba73ddecf125709b5d7/flax/nnx/statelib.py#L170) and [`nnx.State.replace_by_pure_dict`](https://github.com/google/flax/blob/764e1732dcd3b8bf178b9ba73ddecf125709b5d7/flax/nnx/statelib.py#L179) API to convert an [`nnx.State`](https://flax.readthedocs.io/en/latest/api_reference/flax.nnx/state.html#flax.nnx.State) to and from pure nested dictionaries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<script> (()=>{ if (customElements.get('treescope-container') === undefined) { class TreescopeContainer extends HTMLElement { constructor() { super(); this.attachShadow({mode: \"open\"}); this.defns = {}; this.state = {}; } } customElements.define(\"treescope-container\", TreescopeContainer); } if (customElements.get('treescope-run-here') === undefined) { class RunHere extends HTMLElement { constructor() { super() } connectedCallback() { const run = child => { const fn = new Function(child.textContent); child.textContent = \"\"; fn.call(this); this.remove(); }; const child = this.querySelector(\"script\"); if (child) { run(child); } else { new MutationObserver(()=>{ run(this.querySelector(\"script\")); }).observe(this, {childList: true}); } } } customElements.define(\"treescope-run-here\", RunHere); } })(); </script> <treescope-container class=\"treescope_out_f7edccc33c5b43588628e4ee5c70feb6\" style=\"display:block\"></treescope-container> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_f7edccc33c5b43588628e4ee5c70feb6\")) .filter((elt) => !elt.dataset.setup) )[0]; root.dataset.setup = 1; const msg = document.createElement(\"span\"); msg.style = \"color: #cccccc; font-family: monospace;\"; msg.textContent = \"(Loading...)\"; root.state.loadingMsg = msg; root.shadowRoot.appendChild(msg); root.state.chain = new Promise((resolve, reject) => { const observer = new IntersectionObserver((entries) => { for (const entry of entries) { if (entry.isIntersecting) { resolve(); observer.disconnect(); return; } } }, {rootMargin: \"1000px\"}); window.setTimeout(() => { observer.observe(root); }, 0); }); root.state.deferring = false; const _insertNode = (node) => { for (let oldScript of node.querySelectorAll(\"script\")) { let newScript = document.createElement(\"script\"); newScript.type = oldScript.type; newScript.textContent = oldScript.textContent; oldScript.parentNode.replaceChild(newScript, oldScript); } if (root.state.loadingMsg) { root.state.loadingMsg.remove(); root.state.loadingMsg = null; } root.shadowRoot.appendChild(node); }; root.defns.insertContent = ((contentNode, compressed) => { if (compressed) { root.state.deferring = true; } if (root.state.deferring) { root.state.chain = (async () => { await root.state.chain; if (compressed) { const encoded = contentNode.textContent; const blob = new Blob([ Uint8Array.from(atob(encoded), (m) => m.codePointAt(0)) ]); const reader = blob.stream().pipeThrough( new DecompressionStream(\"deflate\") ).pipeThrough( new TextDecoderStream(\"utf-8\") ).getReader(); const parts = []; while (true) { const step = await reader.read(); if (step.done) { break; } parts.push(step.value); } const tpl = document.createElement('template'); tpl.innerHTML = parts.join(\"\"); _insertNode(tpl.content); } else { _insertNode(contentNode.content); } })(); } else { _insertNode(contentNode.content); } }); </script></treescope-run-here><div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtWQtX4sgS/is9mXMXuErkjeDIuQF5OaOO4oyOu3u8naSTtIRO7DQg7vG/3+okvMFxdt3HnV04R6BTXc+uqq/ad4GYuKSmCk5IYHg+ueWeJ9AvyPcCKqjHqogTFws6IgfI8phIW3hA3UkVDTzmBT42YH3sUEHS4Y8q8jmsuDQQ6ZB1Wkx8WGUeg2UdG32be0Nmpg3P9Xg12nqA4l+6CwTAj5rCqSKLCiBjgjBxgAaUpeP1bCbzL+DlPaQD+kiZDfs8bhKehqUD5GPThMW0SyxRRTnDkdowknYItR1YyapFKY8JTMG4Gf/4S3pEA6pTlwowEQ+FN6NNUyY4ZQE1pFgSPY3tenq3F/nx3cyPaT5kIJPDWmBw6gskHXGYwL7vUgNL1+55hiDSTZzgQaKWTKYOa+B5kBcIZBKLBegQCYcGqk3EBYTl1DNJMqU6XiDU8DmYRgS69QmTJmuG5Co3/fjzpicdzEyXwGM2dN2DSIIKavY8j8Fqcuzxfgot6uBdwZJ8tLQsqCEXfcItjw8wM4jKvHEyFR4EEJBce4LS0aZ3KJ9LAR9qoeSK1qpLmC0cdHiIMpLkWdU5EUPOwO+IuAGZK+YMmdRslXXgUEtI/UIC+eUJ3lskJOH4MdMbq5zcD0kgNEYHYbhaHA9IMvJJSvI4WBPkDwMncuPBBhunIg4jM56x8uU6SC2iQArPtt0ofW/DFIPT6ktecoW4YheRERzwOJJSu/C32icT6XSFK1KhmFg1XBwEHyCLY75JZcbzdgDHUJkKf0qBP+H4h2e89m5vUwKYdIRChofKcp1RkMA6WEoeDpWMAqnLxTqJx0BFcAaDR88lw2YPJOWeqe0KJGNU78KCc4t1nZNReH7C+vO2tJ/DmQxYFRMY3mAAGxcocPiSxq+Q4CrzRLLqeCPCUxvoY/JAFhF7kWE+W8wWJQFYQTgn5q0PZZA4ngs1bZGwLN9xFQ7NqCIqMFQTudkCcqyD+Qz8sqV+g85LZLeRx4DapAEInUzr9CohqiEX68StVnUCmU0WtDLC18FGeVEJTmdlDY5rN/h2JouysDDrrieL/laZoUfXJZuY9wOCbTgtbH13GIyVJQcHyVrIs7bRD1EADYcYfWKmUujfqbkOcuvmTVP6JQ3DXlJFiZ9yRd1I/JnqLW/aqmTpD1BSxlEKHvJABtD3oKMSvkEuDV5PbJgKoaB0WAOCbWf8daTOzRPkQaxLUWlwa1EeiFuP3crjvyG1nkslNVeU2bQxVOg3qx9FfFVFadUAcxswUKRGmNBPv1Ea1EN/og+FAPCxqQDNH286tApSVqjAkYBANxP/RLIFU1kBsIkTDKeCYhf1JgPdcwN0NhTSXhM1op3w6U8gMdJjovcBjEaVdwD9xQlhJ2YCtlMcEHMGYd+SjHwfrB/zaHcIHTNqhQxWrYzyY4MVm8vdfKc6xsGtAX0AHDvbjy2x1D2mdfo5mSt7lkUuuh6NME+m0yYWOI0ZBDYEJ6nFZSlEIi6O2fQ0h2xRNkAEPAZwOu0NxbeZMtMAAkOJ+WZZk1AkekMHvscFZmu8de71CbuVK/Ni9HXvLmxb8Oc0zE+qxC+gmHlrAPo1OWGxqsvzBfBcJnyl6rqQOnEjXUpVA7tGEoYgQOBZ/yEEbWogsNw/0/d30yQeyyJNTE+A7VKLRecFDpaIZ4w5g5S6nZbsqZctCxvZ/AZCH7DtL7NZj8ejnaxLsfnxUjqjhgVzPnlWw3EQ87TNsUkhIEmUzRdNYu8iDw6rTVAG8rNkOLvR4QVQKktBuIRiB67pslYzX6cgo7XSO7XnSR07cCglgN1UPWUzgxK7kSaCXQtUIHUbo4gDRMPFPlS5ryPFb28D2yXMFY2IyAOUA3MrzWvosUnEkivmk9UmVyyPLuryyISe4bBqKlsa4jYTvkTYVj6veG8hR0n0RuMcT1SLewMYT42hHIlU2fECdYRdmGGTqZQaeDC8hn1QDqHyU42asRxAX9iOlQQkRWo28gcOIULeC5AxavR6PWlNT67JKT98CDN0OE/1JsxI/vc/MQQwZGv9lXBgcQBj8p7DjdfG8RVTQY6QATeqaMjdpOxNVfl8b+xZVu5Ah+5XKuyamUr7xNbqWvjqnmuaF36rX4zhb6elaU3tuVd9oGl233tvdpv1xviLpl1+aRxrJ916Q2vZD93OB0cE9RNK7Hzr6Dr3oVv6Mur5Q/rxpHiZPb7uXnw+GV2dPIqPk1arsXNl9y9p/Sjj0KPz4XHTbN9lOvqeNeqa/v37knN/Ren58IS1nY71SWifSvVTXtBaXdZvloxPwyHbuSjeG0F/PLJa7t79g9309m39eNzez3a0PaZdFD9wfpy92LEfMxdmRju2svZpuTFu3+XsjDcZXpTLg2a2NO5cV85s2yeX/UmBdPXHoqHzs7bAmn3ePR0f4WASnA+73eurZmusfTz3u1/MT3t7O3b5snydFxnr/cd7bVQEnh+007J2MtYG9uNFb2d40yPN64ecVTIeTwsXnUlxWNfeP9bv/Jafp53zRjNzM/xY6JWZVf/Q7LROBhrd2R81cw7LOuUd/fP4+m7c4aOj9qcGu7OaTVvsnBk3rlsuVhrH4/q+UymcnLR7+faNZg+6xbv6eUVctkmn0qzXu+38kV242PtiTHStDTH9/H5PO29jjZw0XK3z2Dyzb4Rdqn+0z866R/U+PS+SVv26UW8ZNOM73PMZnA3/pnmUfcz2e1bDEs7kPeuYuBV0rMzpoN08LdVN7f7zZx+LoHczME1MKznrsVL4RO/uS/6Al868L40e5e3B6Lid71318q1mzqifW5c7Hdfz24VWMC5i+760T29I79T1r1i90yXmCSfDq/t2Y5C9avF+r/dQzJWuroKxBhqlUHjHJ5KJ8FgnZGv8L/yZZT82PR86/jwlw5tJVVWfodiNcvZn4PX8XY8TXpWFoCzCi8AbjgczUDKCbcsXmZCCl55MXyCLYZ1cC6A8SBYSp0pwh8eYCsTwiNpYeFwFzr7uYW6qY04FuYSRLjnnBcbGvOa3ZQBIksoCiJX3ZCDlkg4IoN3k9CJ1bR8nA0Cka1ufdlEuk8mEsAOKLyCQZDiObZa7gFSVuXJyEJ1WMHm1qKC3qIWpC4VNeEgSvwkrGwANNsQuVGMKPiPYlEB7Z9F38Z3fV277JGifXvct3+isgiSl9i7q0O8o84dxp1HCnqx7D8pGJnH7hodR6wYlws21X97txcwWFEhMsU1ieVku3bojd3lZWYPyynPPVx4u3u8ptR/ePuTKBxHUzEY/wG9AXqui7S76LR75iifmE00Ceawhz9hh4huTLLxMTSXQbNw6VH5MxDYmflZQ2BQPlYV5rIp+uB964mCBLFqAqXp1foQeHB4zKGMOfI/dW4vdNv34C0W3Tzgj7nPBXUZ8U5bUPFRml716wSQ6zpvF4n6xYOhkH1fKeqGUyeznDbNoFFf02HRJDPq44mCLZ+7wgxqCM7TJg7MJISSckliuh0U+lyzsokLqB1scrMRgnZGyDjc3WpvVszhj5I2cWcwVMkVS2ceWlc/qZaKT/L5VUWq/c3Jsyo2/lFO3+XY6mii13RfQvOAU/mF14cdElCgvrhDzDa9XK5Sl/9YoNQRd8DMNoOHRx5AjcqgJ+Q/ND80nJjlK/apD/zdw7ss/tuTGc+EJc29rFKP/2ym1LAJUNgBI+YKAr2fcU/y5ux7T2X0EHJRvzLbvptFuP/a/Egbl/gYwKPey6OT+gUErwCBn6FbFKGSJlS8U9q2Sni9ky5VCCef0Qq5glb4vGFQplXBeLxuZjEkKBWLt54oEZ0rlYh7gYDGf/wcGfWcwKPetnTr3Dwz6/3DudwODXjPXvps2+2eGNReGNfg94zrbYdJR7X+tKG6W</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_f7edccc33c5b43588628e4ee5c70feb6\")) .filter((elt) => !elt.dataset['step0']) )[0]; root.dataset['step0'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtfet620aS6H89BaJcQEYkBfBOUdJ8viQTz8aJY0+SyWj1ySDQlGCDAAOAMhWN/u++x54HOK9wHmWf5FT1BehuNEhKdmZ2v9iTsYnuqurq6uruqurbcRBeW1l+E5GT/SDMlpF3c2TFSUz2rTA42Z8n6UVA5iRNSXDhzlzP8Xt+Nxh0+86ATMbefN5zZyMyI73xfLJ/epwtvRj+RnqnHS9NvZvr8LcLP4lzL4xJat1a767CnLQBzidYULrwoql1Z5mAO2E8TwBlDintubcII+BtkcQJxZ5afhIl6ZH1qUf/TK2Fl16GcXuW5HmyOLKcTndAFlO1xGVKNhcXxstVfpbfLEEgqRdfkv1zYOGapHnoe1Hbi8LLGLgIgyACSvMwygnwcAnUMsgnDbdpJVBUmN80nM6gee/Cjq6SayqoKun70YtXixlJgWCc5I2jeeKvsiaQnSVpQNJ26gXhKjuyesv1+5FkvynTSF60yYj+mfLijix3ubayJAqDMmtDqZ0MIEma6fqyqfUoC3m4BBxFkafWMsnCPEyg2bwZ8LDKIW3m+W8v02QVB23OMi3IxPAsAlig4gVBGF8yvfKvkGwYQwu1yTWJ80wU9i4M8qsjaL28jcxB1tRCzuZR8u7Iug6zcIaKU63Wb+0wDsgaSnYcZ3MtZ8l6x1om63Z25QVYtEP/h9WiFWrxhC4k8KqbK1TwNdnAlh+F/tvAy737tFiUeCjRiwXJMu+SSNojevTd8SEbS47zlJDMT5akna7i9hVJIS3z03CZW1Q3bW+5BB48lMBh4uckb2eA4y3s0z38A8VmuSW4sE6sRqNpnZxat3uWBf/NV7GPqFZAMpKG0Mt/Iz+CNMYN7DQAYFkpyVdpbNHUR0inM0+TRcPLkxkAtazGghJcdPwkIC9QlI/yhtNsTgH7bq++mK9BDHmvWxbEWJ3d5CQDPh9UniAyR9pIJSbvLF4QpdWg5Duz1RzGdo7CK8hwtnH9LP6n8EyLuRfHjOWI5NYTVKaFt3z558dPQTOnem0uSf4ElDGMV8kqo8CNay9akRZTQ8BENFFDpDjzMnJBe0PLSubzjOSMj3BuMVTr+MRyBIYlwUN1nClPZZhlyp1FooxIRE5PLLeGiMxZJyLxZX5lta1uhbTbUYkLYkzEfp4VFFmRX1oNM2m3OTXx8dzLrzogd5BZQaxZ4aIs53PL5fxILZ1qFTorizg/c86RKRdYYOSa1gEnb9UhWQeWyxHl1mGFXW4qzH1oYa65sNmmwroPLayrF8b1/yxtWZcta3Zu7rQ3MZhO/qPUz8L46iUB6g1e3ltyQ8f8n7jaR+HyuQe6nXrvnocx+xe/OYk/e0uhlgX1LPdgPnuF9knAi2gARu6tSg1Gzf4kzL4OY5gYGjTrH/9gKgRTVWPdtA4RwTq2XNLul3hFBddCszRtLgAorQwsMyT2JSX2ZQGDfyhAlFw2qqUecOxf0xwahX8tk3eNNQNoWd1ms9DtO0mLi76vyNE6UcYAzGfy1DI08aPGMPnXVlaHZ2RNnOmQRa2VDDGK0lYWUAtv3RDtzhlqTmtqelxAFFz+i/5h3Q5aC/uMdbKntTxZLxuFCqgygG5YqjfoTgEmqlY0fVEGiEqUdMg6siqWQ01wbPIAwcJYC7imLlN0PewdU0l1BJamOgUx6DGuo6pAMbxiV18SHGVR22He3dRZDzlNQYv3LEHEOJu8d2uVSi8kWX6aVFUnhHINSJwswhhsjLTQ4TBuSCpgqrY29HERUBak0a61hYroKIiuNJvClNZ2KsPGBuTC36nZJHLlFFuZBfLkFRrSr/IULG4216u2WzGJiKlLmWJep5ezxme36Z312e0l/jW7a742Tjdo36deBmbV5cNKlCDQHYnBq7kBiE7X7Q6hf6YwRHdG7qALvy/xtzPq4u9ZOUiVaKeW2x2XsueVsakDZBtVWoBQJ8ZWBKpVFPzz6IWXg8MUgw0E7QH/3bTATaRJxUQJjdxANQ2pwQf/HAsQbmRB2sFBU7PQ0uQdwHPAs/BcKEhB7g0j9wbIAWxB6o1Mik8+ybuzN+dyKhSSrzvI/0vi5w20Lt4A7/BP2LLclmTzlRp5V1EtxmgQXoY5NZ5fpOHCS7Gpziis/emc/rFb8NOdj0azPv05n4/mDqE/u77ndH36Mxh2R90x/TnpD0ezgP4c+4Nhf2a3OEHSG438Ls2Z+bOgy366oxnx5zbAUDHpfL0ikBKonI3m+D+K7RF/RMacs9lsxHkYB/Oxx1Mn48mQ/vQHMycYsJ/9iT/pF5zNR7NhwNgJZsFszNifkMAjg4KzvYI7n0TRK/CigKXRlGVoTgt4JvPwsuKzBDDifB+TJ4AvRjiqe9DI1G1pWdyFCTMY2cKg9GUYwVYxoAuF4MYZhZa0kCsI7cLApZ28jfyrxmDwOQYNmvZ0z6BIUBR0xBHlhv3A/5rTzTRHjkaz0rE4XTSCC9rs46zQ0zOnZZX/nbeUDJemutWMD4Jx3qz4carQOxihgGkAquwXDqetd3mfm/2K9VIhJaCU2SbMvvO+Y85jU+7oFYnLA999m89ADieF7mBA5QH/NiXKD2pEELEridg9l+ff2mYpGsatNOVmrLqyzpuaCV665tCGz6AdwJG5UQTNWhCbCuRiCCyAP96yKm1XFalpmv5QzbVxZn7/lmMidu/fPA/LMpa1ueXaD2o65w/RdKVETY1Q130298fNXdXd0un0JuITV4COL5/D+JA3Ve0dAXeKUTDN7nloP71vc9+3wR/c5A9u9I0dbFumK/U/916Y9ZnnstVZ9t6iOY8t597N6fzxmtPZJHfnwc15b7KG5pSbrjRbRAM31bYtfPz72x7GAisklDz8o6qAQZsEp5JKKU1392B90bysbYakZS89cBZyHh22K+G397Io1V4lss/oWHsum50cBPhZARswtZLgo/X54a1PfSKsiZuYZFSEbVo0btOSAjf3b4KdlZP63rOkVi/RiX3hpXn2+OYpghaOOZWL7HCh/Ebn1RRotm7L6plzQLCDrRBDDtHHf3ugQFshBxySYvQfgtHDfwfvhwn/DlFfSu9TjjGH8TX68SDQuQetJHdWZgV/brnWJ1o8srSeCuw8XZEtWhiTSy8Pr0mxhHhcrnAKmIV3Ceb2KlCWIHSbTYrX4KpvgdPB8ZgOxc1OtozCvGHbmqnHkMRi5XFFsXiOyWhA0IslwkKZOt6ZQvhcHuNRysuU7pC4SMmSeHl2kcxxtXoVRco8bgj8KWSn1sFBqM95vIdnOXDTsrIwIJwHziVjWQoIVmQIgN/RbTpcOgDbVIELydGhu1oZnafqsGIIrjGRaWxZu4mqZr7eVC4PNt6nVAqqFWu0D+hYWLEOyhmEN045i1RaS/5dayFoxpDoT83NVkt1Fq2YSnWTnzSun2nVOzPbfqWTZ8wwGITnNdU1NLBcf+PAI5tFcrCLAX+0ND6kpWHcK7KzGHYXb81SCpoRbCnuURw8i4PQJ1lDj2SHLB1/ZKAnFFXbgCQG9DM+JNBBWKwlAZIFI4EJW2gW5JzZaMfY59SSgVmHpF5ky8pGy+gsV9mVQKCM2sbYU5WkbrEL1otNM7yaZxzVW4eZfX6uTnwC+MTiUNnbcHlBxyFbW+qR2H392a0B/O5ITSZxAImvzd44L/j4vuUyPBPVbThW21QW7kzC9arNfO/SHqzx9EWzyoJIlT2KeI8a2Y2Fl70lgZWs8qb9IDYvoiR5u1pWuBXrN9YXX1ifcNzwMk5SdBDpaLmhder5qlaHqWq2mmU52Gi07xYqyHi7oGvV9rnmLgpOVdQ6z7HC4Sp+GyfvYoW9GqtBwpMLq5uXdpE9dkGT6NG6M3fbDqKY++zp9q5T0HxID6jq1b2ajRWpsb5rq21ts809pKa97uocEbm44z+d2ppTkUSkQ9I0SRv2j4wXeey3+Txi3NnFdwGwAt4kYSx8D2WD6SNo5FdL4ldWaS/A+PNufozzMPqJbfhuBARjf3R7csvyKJiQnTR/0e3hIcw6N9/PMpJe0807fB8sSTNC8URWowE2bhqSrNjILNqLp585551QQnyJxYMCOtXZh2/Ffu6lb3Hb/Ykl8dv5dUXSm1dgz/p5kj6Kooatb92WRc8q15CXJYQrRCLsMFphqgoBSCcli+SaNJomVa5KqBOEGVQiRttDb8yWdXtX7BWGamT5oxgcB2Tw69RbEGkTeA3xhP2Q20+YMuad3bNVGAWP+D7zr8PLVao1vk+jJaLW21RFZfBiV+oqi7Jq3pM/YTzNExbExkAPfsm7Xemu+sdeRob9EkhKrMA+ZaEiBZSmyZB03noOU5NOWcuQcfAEAgYNAroNncNLiTLsWnRfCbRMkyFvDJA3RsgsgjkgMIBrGTKOGkUrUXxtJ4hk5YZrGAlfkBS3gZQISrImyRX5mtrYuP7wTDKAFanWAU33ynEKuzFtr3K7ldykMFvO2TEDaaakAGxM0Q89SFpS3UWhEQ7jrWTZqYRaogIzv0LfBMfVr9j8sIqz1XKZpDmYQQGd+ZvV7epU7y7QWFILZedENK1sykIrJJdE6OTyrfCY4K/SFEZrNTEjS+rEOLIXo4WSir26pcp2RPDhRk9qanvMsAA2b/Lyy9h/wQ+yeiC+i539lH8l8U6vp1rjPMm96EkSZVq9k+hnPChF6+melxmsOiDVihunCaBSb+M+ugwAAKcElkJnPI6JO3IhpwPWE9hU9Cc1sQRYyRETWWmayHUrf38JJA+oiKEs3OkfK8coipozagWeonCF5F4m7zTJgeZ+Q8LLq7wiuptdRXdzH9HdvIfobjaLjleu/L1FdGXVJdkhYrNGFfmU42Nc7RWOwN9T6xYLvb2rl482WO8gJA1DklS18DMqpxisj3PpDNCezrXvgeeWyUt3jSDxVwvoeB0/JV5OvooIfjVsBmoXx6joZ4ceRMS936VqHlhdPB4hdh8q4FdUsgU8bQ8zvCTUJxQX10nJOld45VT5KirkNuxuQFk0DBJ8xzU9m/JvfGO2EicuY/ZgfuL0hk2omkbSKoiYOV+WG+DlkPOmpSvj/jwjCudZtV75Pn8osAK/kI4DSJv7K7vD+e5+fWd4ZX2Z7YmhJ0AezTJTiUWmujGhQPTWGxBpZoVhc0OdMPlW1zIqzcA3tNMtxygQ3HKPm7d14ZjX77kBSTfAPxckNxxvKv+Y+G5hqa1ShK1SKC3L6biD5nTX6ig8YeIBHuA7LM/lbQ2G8/UyPCRkUJ0wNjThffSMHT5ayAd1tlXK6Qx2aJENLdzG6tAWRk7Z15YgzJ7hdAcY78/0E5RSWJcOwOBTaiOwct6iIHFwInsI8kD85eaBeqpzyF2Icj6RzBHUQDY2OdqeipsS/mYzPB/GyqnfwmPVEYHROcVDS7fS+gm3ChhskVoLrvyUHATAp4bzWSGv86rSiUDjiWIN8wHA+hNdv7GOrE8+KbNr6Bl2sssRBG1uuccO973Kup2io4oOmn8yYyq49mKfPElWcS7r3kONKm4RCd0C++ZAatzCwMHU0s6hYMIgUmFrtVox2FT1LUcEmY/TE8lQw+Xiyjqs9qnKploNtZqa7FTe21t5h66pcKfTQpkBlUaN0Jqb8WsWnrXPGdhbb+uXju+04VbTnZNCHMb9CFpplbLkecPaq+7aUMaBdb3aPsSJov7PBpXFIergpPRpavW1VlvXm7UVxblWdXW9SVeVj/UWPV1v0tJaHV0bdXRdr2MoJNRQs5Sam5CN6rmzuijm61pXyvUmpdyrLaFuslb/MQzhnSiMyc/cKXGnGwCzPE3ekprF/DrKT7wlAme/rryUbIX+S0JNLXuBC7z27zrF7m2c2HhlTXtO2EJ6cVSsjSZZyzD6bYMpcr+UddA9t/70JzRTcW/CBgxpXK1DqZtSTfOoWzuPuh/n0Y/z6IZ59PTDzaN7u02ebs3k6X6cPP/Qk+fpB5g86d9yMKy8UYXkSpyiEZN34re6X0nKwAndFORo8qVswUJdpK2kdM8oGW452R4ZkyN3lUVaPSJYSIJtTXiBy1p438FN3TEH6idLNxLR4AMNwIiJDG/5cKeGE/Yy1k0NlioQ3OeM19cgPv+JqadyRy5j/OU9SucAuacOt6fKFCVFtyUsw204cixzsxUmDs5lqyiXYt4fLLIiFocZkTL+Dx10p3CKxXlTQ+KU3vSe4RsF6U6NoqC2Yz6/b6pyl0Koxm9K6eM4f8qcpXa7WvFNi0sFDGXW2j7M86BeEqyiVabC09ufChz6JePp9Ss/Pxf0ppXjrkkaaNd4yYiHnO/KNv3yHjIkcGBY8xEdBfN5/2BIp+VikTpcGtRaHn9/fxWBnieLED6NKnLzcBW52UlFttqruo7ICBuVpFrDhymJgvjHURJxz5oeH21ZxjBni7NTBi3PNy5QlldiYjxfuieo3IKFocs4eAIOZ1C7DBiE13ZTvUsxjClVaV1OX1FhBd+fPqWsoLEVP7HAV5KmF3gWq5G2dDGqPTWCFiuRO8Di1cBf05uBqTcv7gaugc5TL85ws/n3aXjJAgB5soQxYF5HHyauF2myJGl+07DDhXdJ2ilB7Q/jS7zihe65ASEFdnMHAu22uIC0/VuSLJCAuyMi3jbWphchZ2CZUMz+cm0LcbPmUEX92vciv3HtpQ2tXLSaP7uVl4nvlmtxKlCmVLTEbqQYeA0tcT0tPUiAMgPzwBbKwvB3biQZXJETvScZhWNi1a4uPutC53f8fgtlFbX+7LYy8N/RCxw7A7LAcRbqKypcQ++vyVIit96JXGUXQxR9681IJG/u4KtKNP2FdvSBzy24/r6AfkFhaEiNLp1H+Kmsn9MUoUF4+dUr7Ego/aUUT5Oh2E3Jj+lt2wjndFyogXRnshELdZhdfSQNRTYVDV7b5tJl2YbTGeLgWm3EJmocy1C7RFMcR7n7/fcM0fpop6o3j5omScg9QlzZbBRZJKtjfSfkXovU//T24i3FCLmO8zno22e3IeofVT8zHh9PpNpu40Q3XcsIpYE5jII+YYO8cCJohokVhH2EN7CjzKQRwTIqOtfywqOtTFlqftHD2O4iOfPud99L9U9WKWXUTJMcyOOw2Z44Abm0jbQN4zLXqBSHfWMxqTJ/7KxxJk1tWw/X/HtrcNWw3qjCBuVUZbKDdu6pFjSo6rcPVokSfWvvkkB3UKEK9Kwc/euBhBZsgoGxX6wN2dje9ZCK6hZK2U6oVqISK1qpqbLc+0vKDx0iDBSkyZjB3FKgI6keLTxphUmU9l2zbgcgDbw8TtYkq7fg38tHKAvo+JGXZd+GWd4BiwUs3XieoCj5MwyF5XSf8BAHY7ccbtwmyr0gtqv4FWdKU3xtJUFifZdq47MktnSLs1YWp9F4/e/xZ7dlH7k7e61t4MEHMCqs1RVKX86QeiVD5lFUy2aPadhartiKU5USB2B7xOqzvbUqblcDyHKylNc2zKJg4mQo26Rmn1u2JiemNw+UE0Mu5ETfRbG1zHoxcYA6MYnsWjFxAF1MRXLVw0J7te/4V3i7SsYniHKG2CRfRrQqOx8Dp5DzBMYbfKiEPldBrnPFYK/IAgA6UNVLkrOkMvSh6VY94J66Ul2/bZlfMaFTUrq3FBeqif/rTQ5jz1d4cQMORASGS6EbLZNQ9H71EOTqoItRnAgofMAhl1Msxx+R0qEb6ezC/liwA3F25ew6XvUNQy0e5MdLbToDeqbexUto5BUCtQbfL/k0rvBflL0L6wklIQ+cKvFCo4p7w+XzMmhOVQeCHYeBEtk4EEjZrKfbriGHdnK7a8ji3dv24ptqpqiV4UCVTILkj/I8DWcwmTds2sItuWm1QNw8wZe2PlyAj1M0ztwayGanXkCJ0fzvCT6e1daJ8OxSAOZ8+4D68/Rk8jxpVuMYxTNNtR2saoTtJBBBmEnkOxiosJ4i1Z7qPGgT+YfjAQejKg94H5YGdL9wXdX90AjtHK2rUuIWWPGI0weMQwuadSZmAaCG56Rr1tfP6btuMHFoIx0GgHDjFI31Y3vBPPQYn/QK48snUQjMvFQOBUtLPjG4RS+FuD67FZS4q9IuSNPYCwjqtXFFSMQAJJu/ZidIYeKKMJ2Eoy7xcJAO9RE0B6rg2gAvx1rNLhJKWz6ojbWgdi6XoUZqizDZEk6BL7TvtBQdS9Eu39rGrmbnq9yzQEWDBimaFs3C4Huj7XQGaH+5nS5ZaJdEvE8NLVVPePy2oiylELi24MEp/0q+Z8+wF+XBpB01cKLvKlH0fL6KROUF7U2aXPq32/SYEy0QcDvPVnlK3Kinpub5Uflyh4CiVW6JwuiX8iBHnixNWJBcIsGHgkPlelSeni2waEaJRz8VTPGqZBWV5ZS47FtCvjO8z7Jp2UHSAxr6bVuKSOThaNOCQyXcxR7EpBG4ogBcWpHow6cS36qhLw2dqgTloTPlq0HbiBUrCDI1HsSRyLGUkt6ddiodJyaghuYL3TKUlftamNauljDNEMz/Ok0Wr7hlaj5GqOwOwNtOghdrdn0GXw+nqY13YRwk7zoBuQYPg5ZKgZTL+Wtg8LUrZXOPXoxrHcpF4ee24tT3f5YFwKPgzSrLFywGqBVUS7X+xR0/y16s/0rdPvpISZqx8/wN3YZW5CBjaXVXCbp6dRBbQf9Sq0TT+lx6UuOkcsHJ+y3pmq/AvBdNb5Undk0z0TVL1EO5hvImtC+NLVnXqbYsOEMfowVKF1fJFpayhMb4e+fl/pWkt3qP4ZtSSBCqFw1ztVog+nPMbbxupIQGlOkrpZ/d1ijeXbDEIUgLi0Tcra+Ub7Gy+a0xWgjAv6JOY6tAl2Yj03ggz1ZaxaW4hfjBCq4GHupKvas4kfVBC7WSGwa34gK5KGDrzBJ5dpsTJAvQzXXeU1ou89Mkih4rNtktnXRNJeBzhpSDljUjV951iA+52njFkhfn9p1ehuxI03KexXnyU0jeNW4N6EAzSvy3kBITD5SoJHinHnqvynORrDKqGShTPYYmLj/CfaYgOXnDKYa2WOX+1rLKj1+UMU1gGk5AF+4o65b8sV4a+E1iIt0gqXlxdYB1l0EXpyEvrrVrEovNR0XNdimUCrqePbYWrVjpW9xNummWHoiuWUOrlJFr1sv2IuiBj+1FYHOoC2LmmyeVsusFvOHKnOZ0B0XQRS0FJ8A0AoihnocWEwnEQllBmakmSz4oCJiRvxFRAg2bp5vQ6ZViZXeAni7zcWo51hdfKCKTgQ80YOZ6SRyrDqImLb71wVbuu9VguHeseY67LR2r1RJV13TH4LwZ2TRysFPRB3VF11VWkchdTUv9UrTUN8I539BUvxRNVUDLbfWNwZvXeKP9dnNbifXi92ysX96nsarDyz3a6pd7tFW5OG7XHgHZafpKmEWgz147TjE7TTBbGaFxum1z6Etx1OCBM6nAr06n1VMxCPDwyfGinBVZoVP9RtWL6+oNsJu5MN/sUQRA1ZnHfoIZJDjCYLmmjVvuQ9YrU7nuxPRnt8nKUl/MMh8pUk9UbZNJzUGsCpr+LoD5hpCddPXdFSGRSVfF+OhFOZRbDWwFJMq9X9DFYLjNDkspuUJsfr38UzL3QHH0kGj5FnXFaG/i2wtlPqOtvXuz0dKXZVIFxH3tGrvcNYBZA+G+Bb6jHV324n46GVEhNNUPQlc5OrX622rXPrH6uv4ppR7XsIu32VSus1HZVT4P1BOMpqN+tfU4ttpbK3KwrSKndRUJ43tVpL29IobFTJnE7j5w9cfDll6qTuWtHsmlHdOnuL/IcT8M4VqHmg5/qY15DU1CCrQSx2XBZqmwv+nLPR+yNJMvW7aX4brapZdl4TU5Yg+43ClrYqZdFJsa0BjAqFxYywJV38fkKX+258NdV6veLiRfNGrY/MqCFxxIpBjginCxDMoSp+r1TNUr9Ha/RK9yjZ56CV712rxKPmMh13eL1V+MV70XiT4owd5t4JM/3jUBWoJbIo6kh5ZAc/isq52MLQ4WPWAXae0RHtf3lnYdVHl6ZyPYgkYcWZvbTmc0IItaWMliDWO8vaOtes6VdQXzURMzMBrH84geBravmUrXwm4+bWSQWO05Ch0Sa1V45na3M7B3iu9uC1/X1XmVY4G0kZZr7cjG3qazYcUhL6UTbNQPGaROOWSYmg3Ie9qi819xWqhcY445X0UVX3xHja+rtSArG0c8Sa97V967XAEzHn6xMCiPXQBczWI506X7vGAmEpUVNw3ckcXrWvobN29XoEVwpT3YxHPpxbruJjhl+7tPYvlinWrRir67JkDVXdInhDtdH9gM8MdRCVbff5lW8FjPv1wlxMSvakXdlqWiLat3+PMzvrf8sQDNJjqqJjE7znCd/pEpEaHvpnt3TWqF5VchNQFeJkn+XRKQRrNzlWQ5eJvzOOuIAJS4kRF+To8PwXIOl/np8WGeEpL5MAW001XcviIpOT3GDe4W3WF1sj9PogBf87iIgfL+6TGV0+kxXViy0HI42feviP8W6rBvxLnIk8vLCFEPKZJKnt72ceHNZuAF759+EeVTOduOk5xm2qdvvHWHCsEC5gFCIYM+elwCChB+eXyj3wLPsYau9d//+R9tp+OMXev//V+nMxhaZ//9n/8HUia9FuT9l9MZj8+tOIl/I2ly5A63lC6yxT9cXHKVKTRZQ0pAgn01Dw+Mg0YGFz5qGGjJxnzIxG2gPE+07kWhlxqy9szG/ulLMdUzzeh0OoJvo1ZQleFtDv0gCn3aJw4TPyd5OwMcb7F/WrydxfoVVU72hbo5LWIjc/pewEbdnTKwDvDwKkli8b4GJ5EvefQBh13lVZGGnZPFEu0WpEPSFAQGFiKePCltWfY8wl9eff9dh3rSDSTY4QfEdXqs7nZTGTSAGqIUD4xYaofrmB7HYOyWL2vgoLGlN4q6iGZUqrS/oV3eZAnoz+0+bo7cP7L2n7L4W3HDDrgPGDUEh9ny6E6AZsf6Bi3YQ5aOe7rYswg04rjfsvalxxCQYnSdPXvxLnp08O3wl/Hz7kH0zCfX42+yUTq8SX99djD881+/ebHoPX8yOvjxUXR5+Levf1pdu2+db2eTX73BD5Ofn/7gXw9/++nx01F21X10+Pdvb759cXlyUhRF32vAknhXxgztbQTMffTDV4+r/2d0pMsZAPR233PhHxcLcOBHH5yd/fJwLqSc3e7juQQkC6CQS7sw/QQoywUcTKWHEiAVvKl96EaU1Dn8vqmj5VRpOVtoaaehkCAmqzck0UoJMZV3KSFdmOchkY5lzmQ8mYyd4WQ07LljzPPWWKIpi8dpIZs5iPvFrdYUwxl1+yNn4LjOcNAd9Lsuh6CXXdfRFA8/Yx3OJkN23qBHX6R1e3322Xfp53AEOUP8P/0cw88Rvq3LPidAzXXwEUD2cK+D370J/OWyd3qxOHeEL00OWEIXEroOjObuhL2I28eTD1hCt8cgnC6efaB/sUKGiD7Bhwkdhya4eEJi5NBUTBgNkGUsaUwxeviz2yu+kdikX4BjiUOs9JC+udsDan2k6Lrn56iAytkAENGIqXldiJ1plmhz/qzgvsCh6oWdZx9DQBvBzlQQel0J7XugcqXSlg+FoWaKBKGo28qA31sKcd+/kPPtdf332ILxXoVjmXfnd/IQzIdb+AmTa2VCr8z+mgmzaar/h/WU7k46sp68+NFyOLEvLvOpuRTGAE7x1KiFoZ/FLo7QHiH7VhigrZVeFDPCrB+QmdcLBoPxoO/PyNibjGb9oeOMe34w8AecPfgb6Z2yV4HAXligs3KhcHsEJlKjo1hyR1de1jhlFk3HZONRnCNqDpIA71Zo0sk6wojlpx79AxMe1I6W/vuYmRYvfhdz05BllIWpVQE88pYZZlKr1aAYGo1PjZbr/1oLd6NE6lX6dzaIqQAbZ2e4r703cMbjidXCGbA/7o+HPXxuFj+cbrc77LKcoTvpDobdYfmU7RmCDIfDgeOAXIFSzx2NehOcdPBjPOn33VGPIg+gc43HTl9CBpBxvzcaT3Ba43PvBDJYYaPhYEw/yonUUkseug5QHzEQtzsZdekEiWRHI2fU58jdUXcC09X4HCYU5qXSXsIVolkj/fcbXibDodebjXzHCUi/T+bj7oB4znA06MHAM+j1yuHlj+dAdgesd/VZ73I7Lu1cbsf56D1+9B4/eo8G7/HwifeX69+iHwHlqxcHP0yG79yvrq+fJM5s8NR/fHDoDfzZdXd29Zeno7z76uDw78vFoxe9n585qbO6/uH5IRk9W10Psh9/S/u/3jjDg9nbl+5H7/FDe48wgo3cAcyYznjkTEYw5RTeozFri/foOt3RZNLr9cAxnEy6OLUp3qOR5kfv8aP3+NF7/L29x64/m0/8vkvmvX5/PB/OemDnTvpDrzvrd/vz4e9r3n103P4ZluX/bK+tnGyYuzQZjcb90ZC5PzDe9fuTEXPh3InbH0x6fc1rcyb9CR2Q0ZGCsRucE+54OaPRaNLlXhX6WOC4aY7XeDiBEReGYZyEnP5w7PYYspizKPLABZ8PGkNHhonBEc6m0wXfEecpbD7HhT40HjOvrTcZuOPx4L29tvc2sdlFO88CestnaRW6M9dz/J7fDQbdvjMgk7E3n/fc2YjMSG88n1CbaOdwkwy81XmUgbcORedT7gkkfgrGv9EXEOY6yooDcXDNQN/vFOK8wOz9CuqTKKE7XMR3x8cEWgx9sGda5qQEBlmf/BzmVw0FvSA6T71LfnxW2zbwlH9+zSGwDgJaWXoWhDHfcPK+aFzt5D17ClgcjS0owyffsPD45hkQF9j4JsaUPSKdrFKfPKV3ztTI8FOcS/atA0tD5zc6yWIpqHXmYZqJsmnNAKHMLX2jO7UljELe5gr9fzVOm+E=</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_f7edccc33c5b43588628e4ee5c70feb6\")) .filter((elt) => !elt.dataset['step1']) )[0]; root.dataset['step1'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:absl:`StandardCheckpointHandler` expects a target tree to be provided for restore. Not doing so is generally UNSAFE unless you know the present topology to be the same one as the checkpoint was saved under.\n"
     ]
    }
   ],
   "source": [
    "# Save as pure dict\n",
    "pure_dict_state = nnx.to_pure_dict(state)\n",
    "nnx.display(pure_dict_state)\n",
    "checkpointer.save(ckpt_dir / 'pure_dict', pure_dict_state)\n",
    "\n",
    "# Restore as a pure dictionary.\n",
    "restored_pure_dict = checkpointer.restore(ckpt_dir / 'pure_dict')\n",
    "abstract_model = nnx.eval_shape(lambda: TwoLayerMLP(4, rngs=nnx.Rngs(0)))\n",
    "graphdef, abstract_state = nnx.split(abstract_model)\n",
    "nnx.replace_by_pure_dict(abstract_state, restored_pure_dict)\n",
    "model = nnx.merge(graphdef, abstract_state)\n",
    "assert model(x).shape == (3, 4)  # The model still works!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<script> (()=>{ if (customElements.get('treescope-container') === undefined) { class TreescopeContainer extends HTMLElement { constructor() { super(); this.attachShadow({mode: \"open\"}); this.defns = {}; this.state = {}; } } customElements.define(\"treescope-container\", TreescopeContainer); } if (customElements.get('treescope-run-here') === undefined) { class RunHere extends HTMLElement { constructor() { super() } connectedCallback() { const run = child => { const fn = new Function(child.textContent); child.textContent = \"\"; fn.call(this); this.remove(); }; const child = this.querySelector(\"script\"); if (child) { run(child); } else { new MutationObserver(()=>{ run(this.querySelector(\"script\")); }).observe(this, {childList: true}); } } } customElements.define(\"treescope-run-here\", RunHere); } })(); </script> <treescope-container class=\"treescope_out_277cbde407034bf9addfa8e4969bce7e\" ></treescope-container> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_277cbde407034bf9addfa8e4969bce7e\")) .filter((elt) => !elt.dataset.setup) )[0]; root.dataset.setup = 1; const msg = document.createElement(\"span\"); msg.style = \"color: #cccccc; font-family: monospace;\"; msg.textContent = \"(Loading...)\"; root.state.loadingMsg = msg; root.shadowRoot.appendChild(msg); root.state.chain = new Promise((resolve, reject) => { const observer = new IntersectionObserver((entries) => { for (const entry of entries) { if (entry.isIntersecting) { resolve(); observer.disconnect(); return; } } }, {rootMargin: \"1000px\"}); window.setTimeout(() => { observer.observe(root); }, 0); }); root.state.deferring = false; const _insertNode = (node) => { for (let oldScript of node.querySelectorAll(\"script\")) { let newScript = document.createElement(\"script\"); newScript.type = oldScript.type; newScript.textContent = oldScript.textContent; oldScript.parentNode.replaceChild(newScript, oldScript); } if (root.state.loadingMsg) { root.state.loadingMsg.remove(); root.state.loadingMsg = null; } root.shadowRoot.appendChild(node); }; root.defns.insertContent = ((contentNode, compressed) => { if (compressed) { root.state.deferring = true; } if (root.state.deferring) { root.state.chain = (async () => { await root.state.chain; if (compressed) { const encoded = contentNode.textContent; const blob = new Blob([ Uint8Array.from(atob(encoded), (m) => m.codePointAt(0)) ]); const reader = blob.stream().pipeThrough( new DecompressionStream(\"deflate\") ).pipeThrough( new TextDecoderStream(\"utf-8\") ).getReader(); const parts = []; while (true) { const step = await reader.read(); if (step.done) { break; } parts.push(step.value); } const tpl = document.createElement('template'); tpl.innerHTML = parts.join(\"\"); _insertNode(tpl.content); } else { _insertNode(contentNode.content); } })(); } else { _insertNode(contentNode.content); } }); </script></treescope-run-here><div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtWQtT4sgW/is9maoBrhITnoJK3YC8nFFHcUbH3S23STpJS+jETgPilv/9nk6CPETH2XHn7guqBDqnz7PPOd9pd0Mx9UhNFZyQ0PQDcsV9X6DfUOCHVFCfVREnHhZ0THaQ7TORtfGQetMqGvrMDwNswvrEpYJkox9VFHBY8WgoshHrrJgGsMp8Bst9bA4c7o+YlTV9z+fVeOsOSn71PSAAftQSbhXZVAAZE4SJHRRgy6LMyXrEFlWUM10phJGsS6jjwoquFiUbJjAFnR+2JV+yYxrSPvWoAM3xSPgPtFnKBKcspGY2pHckfpqoe7+7Fbtn98E9WT5iIJPDWmhyGggk7dtL4SDwqImlx7Z8UxBpPSd4mKql05m9GjgU5IUCWcRmIdpDwqWh6hBxCt4+8i2SzqiuHwo1eg6mEYGuAsKkyYYpucpNP/2y7kkHM8sj8JiNPG8nlqCCmj3fZ7Canvh8kEGLOvjnsCQfLS0LasrFgHDb50PMTKIyf5LORPEFAelHT1A23rSL8rkM8KE2Sq9orXqEOcJFe3tIkyTPqs6JGHEGfkfEC8lcMXfEpGarrEOX2kLqFxHIL/fwfkJCGk4Vs/yJysnNiITCYHQYhavF8ZCkY59kJI+dR4KCUejGbtxZY+NMxF5sxjNWvlwHqUUcSOE7jhdn5VWUOXBaA8lLrhBPbCIyhgOeRFJqF/1WB2Qqna5wRSqUEKumh8PwAyRnwjetPPC8GsIxVGbC7zPgTzj+0Rmv7W6tSwCLjlHEcE9ZLh8KErgPlpLbPUVTIHW5eEziM1ARnMHg0XPJsN4DablnZrsCyRiXsaiOXJn+cAjr0QGK6spbHL2kbSskuMp8ka66/pjwzBr6hBxUhfhbCxw1eNn2nCKUVcRZFJnXi3pREoAZhHNiXQVQ3ojrexbhi4Rl+U6qa2RHFVGBoZzIzTaQ4z7Yz8AxT9RlsGqJ7Cp2GVBbNASh01n9XSVENeThPvGq1T6B1CYLWpnRa2etvLgGZ3VZhJNare3MZVEWVea+58ti/qTMyOePJVuYD0KCHTgu7PHuKFwrSy4O07WIZ22tH+IQmy4xB8TKZNB/MnMd5Nb1m2b0SxpGzaSKUj/nin0z9f9Ub3nTk0qWfoCSMo5S8IiHMoCBDy2V8DVyafh6YqNUiARloyIQPnXGX0fq3DxBbsVjKSoNr2zKQ3Hlsyt5/Nek1nOppOaKMpvWhgp9t/pxxFdVlFYNMXcABMVqRAl9/53SoB4G0/5ICEAf6wrQ/PG6Q6sgZYUKHAnIcj3xz0QvWMoKME0dYjgVFHuoNx32fS9ExyMh7bVQI94Jn8EUEiM7If0BgMy48g6hwbhQwwECMgHbKQ6J9QBN3xJNvnceH/N4d4QdNbVChqtWxvmxxor15W6+U53g8MqEPgCOfdiPbbHUPWZ1+jmZK3uWRS66Ho0xT2ezFhY4ixkENkInmcVlKURCLo7Z7DRHbJEeIgIeAzyd9Ufi20x50AACQ4n1ZlmTSCR6Q4eBzwVmj3j3uT8g7EquzIvR1727sG3Bn7Mw36sSwIBi1pUJ8NfihCWqLg8YwHOZ8JWq60LqJI10KVVN7JlpmIIAguvBbYTa1FBguf9B3z9Mkz5AIai7sSaWL8B2qcWi80IXS8QzwZxBSl3NSvbMy7aNTT2/hjAAcPvbw7DHk9lO1qXE/GQpq6lRwZxPlNVoHsQ863BsUQhIWs8XLeJsIh/OqkOQBulZMt3N+OwCKJWVIFpCif8eqfKoZL6aS5eCOTPmXp240Yn0PBxA8fk6gPv26vy0hBi2RTIiInILWWo9SfMaeqwTkRgaE8wnnnWuWB4p1OVRBj3DYdVUtjRcrSd8ibAn+bzifYIc8dAbg3M8VW3uD2FsNEdyllFlIwrVMfZgtkxnMmrow1AZtSc5HMpPNe6RcjB8YZdUUiiDMg+jeOgSIuS8Tiao0ev1pDU9uSan7+ghzLbRmNObMjP963+TzmzKjvc7u/TiXMTk/YOXrE2Sq5+CBiUn5GYVjbiXli2jKp9vTXzbzu30oSmVCpuWVmkfOkbdiF7dE8Pwo2/10wn87bQMo2k896oPDcMZ+O+tbrPemHwxjLMvjQPjsFtvGC3nttv54IqwfkiJk2/tX+Q+dEtfxr1gRD8eFs/0g4vu6efD8fnhnfg4bbUaG+fO4IzW9zWX7p+MDppW+1rr9LfscdcKbt6X3JtzSk9Gh6ztduxPwvhUqh/xgtHqskGzZH4ajdjGafHGDAeTsd3ytm5unaa/7fQPJu1tvWNsMeO0+IHzA/10w7nTTi3NOLB156jcmLSvc47mT0en5fKwqZcmnYvKseME5GwwLZBu/65o9vlxW2DDOekeTfZxOA1PRt3uxXmzNTE+ngTdL9anra0Np3xWvsgLzX7/8cYYF4HnB+OobBxOjKFzd9rbGF32SPPiNmeXzLujwmlnWhzVjfd39eugFeRp56TR1C5HHwu9MrPrH5qd1uHQoBvb42bOZbpb3uh/nlxcTzp8vN/+1GDXdrPpiI1j89LzysVK42BS33YrhcPDdi/fvjScYbd4XT+piLM26VSa9Xq3nd93CqdbX8xp32hDTD+/3zJO2tgghw3P6Nw1j51L4ZTqH53j4+5+fUBPiqRVv2jUWybVApf7AYOzEVw29/U7fdCzG7Zwp+9Zx8KtsGNrR8N286hUt4ybz58DLMLe5dCyMK3k7LtK4RO9vikFQ1469r80epS3h+ODdr533su3mjmzfmKfbXQ8P2gXWuGkiJ2b0ja9JL0jLzhn9U6XWIecjM5v2o2hft7ig17vtpgrnZ+HEwM0yqDo7k2kU9GxTsmW9Sv8ech+bPkBNOJ5SkY3hqqqPkOxGefsL8Dr+TsYN7rCirBSDOOANxwPZqJ0jKaWLxghBc98mb5AlqAtuRZCeZAsJHyUmAtPMBWI4TF1sPC5CpyDvo+5pU44FeQMJq30nBcYm/Ca32IBTkgrC9hS3l+BlDM6JABC07MLzkf7OBkCUHy09X4T5TRNi+AAFF9ABuloSlovdwFAKnPl5Hw4q2Dyyk9Bb1ELUw8Km/CRJH4TVTaAAGyEPajGFHxGsCXx78ai75K7uK/cwkksPbuGW75oWQUvSm037tC7lAWjpNMoUU/u+7fKWiZJ+4aHcesGJaLNtd92txJmiwo8wszKc89XHi5epCm1d29vc+WdGNPp8Q/wBJDXquhpo7/HxudtS81HhxTyWUOemr3UN6ZNdG2ZSaGHuWZP+SmV2Jj6RUFRm9tTFgafKnp3M/LFzgJZvADj6+qgBl01OjhQmFz4nri3lrht9vGHxqsPU/NzwVq6TlVqR4CLEsLNJbplmKrU0DrtlzCk8qOj9VNKGvviqM3IvzN6L43lC2I1IJwR77loLSPoGUtq7SkPd9rYrJRxv1wh+VylULTzFd3WyxVNK1cqhGwXV/VYdxeu1K7xrRohWmR7Phb5XLqwiQqZr9v8GHWvVbKSL1swDfY1oucKpl3CBV3D2xbRSnYu39fzSu0Prihri+V6X7zzxM4T/njniJ0XH4eV5Nj8xgR6IvY/MLvi4/kN+TXb8Ir1cemfRVCFoJd/piG0bXoXcUQutSDroIWj+dwnB8LfdWb/Ac595uP+r9oIfkzbfsXCH6uU+weAqtzLopP7F1T9GXIp922gKve3BVXE1i29tJ2HQV8rFPM6IKuyTQr2dsXMl4p9/U8BqvSSXsnlynqltL1dsLXt7XLeKtsAsHLlMraK2r+g6s8FqnLf2vdz/4Kqv4ZzXwCqXvMU/22a9u912MMOi45r/wOOvCUk</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_277cbde407034bf9addfa8e4969bce7e\")) .filter((elt) => !elt.dataset['step0']) )[0]; root.dataset['step0'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>\n",
    "\n",
    "\n",
    "\n",
    "<div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtfet620aS6H89BaI4IWmRFMA7RUnz+ZKLN7bjsZPJZrX6ZJBoSrBBgAFAmYqG/8+8xzkPcF7hPMo8yanqC9DdaICU7OzsfrEnYxPdVdXV1dXdVdW3Y8+/tpL0JiAn+56fLAP35sgKo5DsW753sj+P4guPzEkcE+9i3B16Trc/tYnT6c3mA7fn2O7II/Zg3ulOne7+6XGydEP4G+mdtt04dm+u/d8vZlGYun5IYuvW+nDlp6QFcDOCBcULN5hYG8sE3PbDeQQoc0hpzd2FHwBviyiMKPbEmkVBFB9ZX7r0z8RauPGlH7amUZpGiyPLbnf6ZDFRS1zGpLo4P1yu0rP0ZgkCid3wkuyfAwvXJE79mRu03MC/DIEL3/MCoDT3g5QAD5dALYF8UncaVgRF+elN3W73G3cu7OgquqaCKpK+G71wtZiSGAiGUVo/mkezVdIAstMo9kjcil3PXyVHVne5/jiS7DdlGsmLNhnSPxNe3JHlLNdWEgW+l2dVlNpOAJLEia4vVa1HWUj9JeAoijyxllHip34EzeZOgYdVCmlTd/b+Mo5WodfiLNOCTAxPA4AFKq7n+eEl06vZFZL1Q2ihFrkmYZqIwj74Xnp1BK2XtpA5yJpYyNk8iD4cWdd+4k9RcYrV+r3lhx5ZQ8m2bVfXchqtd6xltG4lV66HRdv0f1gtWqEmT+hAAq+6uUIZX+MKtmaBP3vvual7lxYLIhclerEgSeJeEkl7RI/etGnKhTudxuRaAhiMOi5l5/iQDTbHaUxIMouWpBWvwtYViSEtmcX+MrWo8tbc5RKYdFFEh9EsJWkrARx3UTvdwz/AV5Jagk3rxKrXG9bJqXW7Z1nw33wVzhDV8khCYh+Ggd/JzyCuUR17FQBYVkzSVRxaNPUR0mnP42hRd9NoCkBNq76gBBdQJ4+8Qlk/Sut2ozEB7M1eeTHfgpzSbicviLE6vUlJAnzeqzxBZI60kUpIPli8IEqrTsm3p6s5DP4chVeQ4Wzj+ln4X8IzLeZOHDOWA5JaT1CZFu7y9XePn4LqTvTaXJL0CWirH66iVUKB69dusCJNpoaAiWiihkhx6ibkgnaXphXN5wlJGR/+3GKo1vGJZQsMS4KH6tgTnsow85SNRYKESEROTyynhIjMWTsg4WV6ZbWsToG001aJC2JMxLM0ySiyIh9adTNppzEx8fHCTa/aIHeQWUasUeAiL+cry+H8SC0daxU6y4s4P7PPkSkHWGDkGtYBJ2+VIVkHlsMR5dZhhV1WFebctzDHXNi0qrDOfQvr6IVx/T+Lm9Zl05qemzvtTQi21exRPEv88Oo1Aep1Xt57ckMnhb9xtQ/85QsXdDt2P7zwQ/YvfnMS37lLoZYZ9SR1YcJ7gwaMx4uoA0bqrnINRs3+wk++9UOYOeo06+9/ZyoEc1l93bAOEcE6thzS6uV4WQXXQrM0bc4AKK0ETDck9pASe5jB4B8KEESX9WKpBxz7tziFRuFfy+hDfc0Amlan0ch0eyNpcdb3FTlaJ8oYgPlMnlqGJn7UGCb/0srq8IysiTMdMqu1kiFGUdrKAmrhruui3TlDjUlJTY8ziIzLf9E/rNtBa2GfsU72tJYn62U9UwFVBtANc/UG3cnARNWyps/KAFGJkg5ZR1bFcqgJjk0eIFgYawHX1GWyroe9YyKpjsDSVCcjBj3GsVUVyIZX7OpLgqMsajvMu1Wd9ZDTFLR4zxJEjLPJR7dWrvRCkvmnSVV1QihXj4TRwg/BxogzHfbDuqQCpmprQx8XAWVBGu2aW6iIjoLoSrMpTGltpzJsbEAu/J2aTSKXT7GFWSCN3qAh/SaNwSRnc71qu2WTiJi6lCnmbXw5rT+4jTfWg9tL/Gu6abw1TjfoAMRuAmbV5f1KlCDQXwnB7bkBiHbH6Qygf8YwRLeHTr8Dvy/xtz3s4O9pPkjlaKeW0xnlsueVqVEPqWZUaQFCvZyaIlCtouDAB6/cFDyqEGwgaA/476YJfiRNyiZKaOQ6qqlPDT7451iAcCML0g4OGpqFFkcfAJ4DnvnnQkEycu8YuXdADmAzUu9kUnzyiT6cvTuXU6GQdN1G/l+TWVpH6+Id8A7/+E3LaUo2X66Rm4JqMUY9/9JPqfH8KvYXboxNdUZha1/O6Z9aE3468+Fw2qM/5/Ph3Cb0Z2fm2p0Z/ekNOsPOiP4c9wbDqUd/jmb9QW9aa3KCpDsczjo0Zzqbeh320xlOyWxeAxgqJp2vNwRSPJWz4Rz/R7FdMhuSEedsOh1yHkbefOTy1PFoPKA/Z/2p7fXZz954Nu5lnM2H04HH2PGm3nTE2B8TzyX9jLO9jLsZCYI34EUBS8MJy9CcFvBM5v5lwWfxYMT5MSRPAF+McFT3oJGp29K0uAvjJzCy+V7uyzCCzWxAFwrBjTMKLWkhVxDahYHLWvQ+mF3V+/2vMKrQqE32DIoERUFHHFJu2A/8rzGppjm0NZqFjsXpohGc0WYfZ5mentlNK//vvKlkODTVKWZ8EozzRsGPU4XexggFTANQ5VnmcNb0Lj/jZr9ivRRICShltvGTl+5L5jw25I5ekLg88N21+QzkcFLo9PtUHvBvQ6J8r0YEETuSiJ1zef4tbZasYZxCU1ZjlZV13tBM8Nw1hzZ8Bu0AjsyNImjWgthUIBdDYAH88aZVaLuiSE3T9KdqrsqZ+eNbjonYuXvz3C/LWFZ1y7Xu1XT2n6LpcomaGqGs+1T3x+qu6mzpdHoT8YnLQ8eXz2F8yJuo9o6AO8UomGb33Lef3rW579rg927yezd6ZQfblulI/c+5E2Z55rlsdea9N2vOY8u+c3Paf77mtKvkbt+7Oe9M1tCcctPlZoto4IbatpmPf3fbw1hggYSSh39UFTBok+BUUiml6Tb31hfNy9pmSFq1pQvOQsqjw7VC+O2jLEq1V4nsMzrWnstmJwcBflbABkytxPtsfX5661OfCEviJiYZZWGbJo3bNKXAzd2bYGflpL73NCrVS3RiX7lxmjy+eYqgmWNO5SI7XCi/4XkxBZqt07S65hwQbH8rxIBD9PDfLijQVsg+h6QYvftgdPHf/sdhwr8D1Jfc+5RjzH54jX48CHTuQivJnZVZwV9ZjvWFFo/MracMO41XZIsWhuTSTf1rki0hHucrnAJm4V6Cub3ylCUI3WaT4jW46pvhtHE8pkNxo50sAz+t12qaqceQxGLlcUGxeI7JaEDQiyXCQpk63plC+Fwe41HKy5huobiIyZK4aXIRzXG1ehUEyjxuCPwpZCfWwYGvz3m8hycpcNO0Et8jnAfOJWNZCggWZAiAL+k+Hi4dgG2owJnk6NBdrIzOU3FYMQTXmMg0tqzdRFUyX1eVy4ONdymVgmrFGu0DOhYWrIN8BuGNk88ihdaSf5daCJoxJPpTo9pqKc6iBVOpbPKTxvUzrXpnZtsvd/KMGQaD8LykuoYGlutvHHhks0gOdjHgz5bGp7Q0jHtFdhbD7uItWUpBM4ItxT0KvWeh589IUtcj2T5Lxx8J6AlF1TYgiQH9jA8JdBAWa0mAZMFIYMIWmgU5ZzW0Y2rn1JKBWYfEblCTlY2W0V6ukiuBQBmtGWNPRZK6xS5YzzbN8GqecVR37Se183N14hPAJxaHSt77yws6DtW0pR6J3bcPbg3gmyM1mYQeJL41e+O84OO7lsvwTFS34VgtU1m4MwnXq6r53qU9WOPpi2aFBZEiexTxDjWq1Rdu8p54VrRKG7V7sXkRRNH71bLArVi/sb7+2vqC4/qXYRSjg0hHy4rWKeerWB2mqslqmqRgo9G+m6kg4+2CrlXXzjV3UXCqopZ5jgUOV+H7MPoQKuyVWA0SnlxY2by0i+yxC5pEj9adudu2EcXcZ0+3d52M5n16QFGv7tRsrEiN9V1bbWubVfeQkvbalDkicnHHfzmtaU5FFJA2ieMortd+ZrzIY3+NzyPGnV18FwAr4F3kh8L3UDaYPoJGfrMks8Iq7QUYf+7Nz2HqB39jO8LrHsHYH92e3LRcCiZkJ81fbP+4n978OE1IfE337vBtsCROCEUTWfU6mLixT5JsH7NoLp5+Zp+3fQnxNZYO+mcXJx++VfuFG7/HbfknlsRu+7cViW/egDk7S6P4URDUa/rWblnyrG51eVVCeEIkwP6iFaZqEIC0Y7KIrkm9YdLkgoDanp9AHUK0PPSmbFq3m2ynMNQiSR+F4DYgf9/G7oJIW8DNtCP2Q248YceYt3VPV37gPeKbzL/1L1ex1vIzGioRdd6mJyp/F7tSV1mU9fKO/AnLaR6xCDZGefBL3upKt9Q/dhMy6OVAUmIB9imLEymgNE2GpJPWC5iXdMpahoyD5xMwYuDRPegcXkqUYdei70qgeZoMeWOAvDFCJgFMAJ4BXMuQcdQQWo4y07aBSCauv4Zh8BWJcQ9IjqAka5JckW+pgY2LD88k61eRahnQZC8fpLAT0/bK91rJTQpT5ZydMZCmSQrARhT9xIOkJcUtFBphP9xKlh1JKCUqMNMrdExwVP2GTQ6rMFktl1Gcgg3k0Wm/UdyrTvXuAi0ltVB2SETTyoYstExyUYAeLt8HjwmzVRzDWK0mJmRJPRhbdmG0OFK2UTdX2baIPNzoSQ1tgxkWwCZNXn4e+M/4QVYPxHe2rZ/yryRu9HqqNU6j1A2eREGi1TsKfsFjVLSeznmewaoDUi34cJoACvU2bqJLAABwcmApbsaDmLgdF3LaYDqBQUV/UvtKgOUcMZHldolct/z3QyB5QEUMZeE2/1A5Q5HVnFHL8BSFyyT3OvqgSQ4093viX16lBdHd7Cq6m7uI7uYjRHdTLTpeufz3FtHlVZdkh4iNElXkU84Mg2pvcAT+kZq2WOjtplw+2mC9g5A0DElSxcLPqJxCMD7OpQNAezrXMxfctkRet6t70Wy1gI7XnsXETck3AcGveo2B1rIzVPSzTY8p4sbvXDUPrA6ejRBbDxXwKyrZDJ62hxleEuoTiouLpGSdKrxyqnwJFXLrtY5HWTQMEny7NT2Y8gPfla0EifOAPRifOL1hE6qmkbQEImbO1/nudzneXLVuZdycZ0ThPKu2K9/kDwUW4BfSWQBpZ39hazjf2q9vCy8sLrMNMfT4x6NpYioxy1R3JWSI7roCkWYWGDY31AmTb3Eho9AMfDc73W+MAsH99rhzWxeOefGeG5B09/sLQbLibFP+x8R3E0tt5iJs5kJpWnbb6Tcmu1ZH4QkTD/D03mF+KG9rJJwvluEJIYPq+KGhCe+iZ+zk0UI+pbOtUna7v0OLVLRwC6tDWxg5ZV9bIjB7hqMdYLw/049PSjFdOgCDR6mNwMphi4zEwYnsIcgD8cPqgXqic8hdiHw+kcwR1EA2NtnahoqbHP6mGp4PY/nUb+Gh64DA6BzjiaVbafGEWwUMNkstBVd+Sg4C4FPD+SyT13lR6USU8USxhvkAYP2FLt5YR9YXX+TZJfQM29jl+IE2t9xhe/teYdFO0VFFB80/mTHlXbvhjDyJVmEq6959jSpuEQndAvvmQGrczMDB1NzOoWDCIFJhS7VaMdhU9c1HBJmP0xPJUMO14sIirPapyqZYDbWamuxU3ltbeYeuqXCn00KZAZV6idAa1fglq87a5xTsrffl68YbbbjVdOckE4dxM4JWWqEsed6w9opbNpRxYF2utvdxoqj/U6GyOEQdnOQ+Tam+lmrrulpbUZxrVVfXVbqqfKy36Om6SktLdXRt1NF1uY6hkFBDzVJqVCEb1XNndVHM17WulOsqpdwrLaFsslb/MQzh7cAPyS/cKXEmFYBJGkfvSclKfhnlJ+4SgZPfVm5MtkL/W0RNrdoCV3drf+gUu1c5sfHKmjacsFX07JxYC02ypmH02waT5T6UddA5t/7yFzRTcWNCBYY0rpahlE2ppnnUKZ1Hnc/z6Od5tGIePf108+jebpOnUzJ5Op8nzz/15Hn6CSZP+rccDMuvUyGpEqeoh+SD+K1uVpIycEI3BTkafB1bsFAWacsp3TFKhvtNtkfG5MhdYY1WjwhmkmD7El7hshZednBTdsaB+snSdUQ0+EADMGIiwys+nInheL2MdVOCpQoENznj3TWIz39i6qnckfMYf36J0jlA7qnD7akyRUnRbQnLcBWOHMustsLEqblkFaRSzPuTRVbE4jAjksf/oYPuFE6xOG9qSJzSm9wxfKMgbdQoCmo75vPLpgoXKfhq/CaXPo7zp8xZarWKFa9aXMpgKLPW9mGeB/UibxWsEhWeXv2U4dAvGU+vX/75laA3KZx1jWJPu8NLRjzkfBf26OeXkCGBA8Oaj+gomM/7B0M6zReL1OHSoNby+PvHqwj0PFmE8GlUkZv7q8jNTiqy1V7VdURGqFSSYg3vpyQK4p9HScQla3p8tGkZw5xNzk4etDyvXKDML8zEeL50SVC+AQtDl6H3BBxOr3QZ0POvaw31IkU/pFSldTl9RYUVfHf6lLKCxlb8xAJfTpre3pmtRtaka1NrEyNothK5AyxeHPwtvTeYevPi5uAS6DR2wwR3mv8Y+5csAJBGSxgD5mX0YeJ6FUdLEqc39Zq/cC9JKyao/X54ife70D03ICSv1tiBQKslbh9t/R5FCyTg7IiIV4216DXJCVgmFLO3XNeEuFlzqKJ+O3ODWf3ajetauWg1P7iVl4k3y7U4EihTylpiN1IMvISWuLyWniJAmYF5UBPKwvB3biQZXJETvUUZhWNitVZcfNaFzm8Afg5lZbV+cFsY+Df09sZ2nyxwnIX6igqX0PspWkrk1juRK+xiCILn7pQE8uYOvqpE019p5x743ILr7wvoFxSGhtTo0nmAn8r6OU0RGoQ3X73BjoTSX0rxNBmK3aP8mN7FjXB224EaSDcqG7FQh9m9R9JQVKOiwTvbHLosW7fbAxxci43YQI1jGWqXaIizKJs/fs8QrY92pLp61DRJQu4R4kJno8gCWR3LOyH3WqT+p7cXbylGyLHtr0DfHtz6qH9U/cx4fDyRaruNE910zSOUBuYwCvqEDfLCiaAZJlYQ9hHez44yk0YEy6joXMszj7YwZan5WQ9ju4vkzM0fvpfqv1illFEzjlIgj8Nma2x75LJmpG0Yl7lGxTjsG4uJlfljZ40zaWrLur/m31mDi4Z1pQoblFOVyQ7auada0KCqz++tEjn61t4lge6gQgXoaT76lwMJLaiCgbFfrA3VsL3LIRXVzZSyFVGtRCVWtFJTZbn355TvO0QYKEiTMYO5pUBHUj2aeMwKkyjtTaNsByANvDyO1iQpt+A/ykfIC2jPAjdJnvtJ2gaLBSzdcB6hKPkjDZnldJfwEAdjVxxWbhPlXhDbVfyGM6UpvraSILG+S7Xx0ZKadIWzVhanUX/7n+GD27yPbM7eaht48HmMAmtlhdJ3NaReyZB5FNWqsac2alqu2IpTlBIHYHvEyrPdtSpuRwNIUrKU1zbMomDiZCjbpFY7t2qanJje3FNODDmTE301paZllouJA5SJSWSXiokD6GLKkoseFtqrPXt2hVerJHyCyGeIKvkyokXZzTBwCjlPYLzBZ0zoWxXkOlUM9oIsAKANVb0kKUvKQx+abpUD7qkr1eXblvn9EjolpXtLcaGS+L/e5DD2fIO3NuBARGC4FLrRNAlF71f3QS4OuhjFCYDCJxxyOcV8/BEpbbqRrpbZHwt2HK5WOLiO93zDUIun+PFGm3afHqh38AYaeYVArcGPSz6NK/xnZe/CekRJyAOnSjzTqOzScPm8DJpTxYFgx2EgRzYOBFI26+k1x5BDO3mtY8ji3bvmhjfFTFErw4EqmQRJH6Vp7E9hMq/XaAs35abVAnHzCN/h+nQBPk7ROHNrINVOvYASo/l/RPi0VksnwrNzAZjzawfUn6fHkudRoxjHyB5xKu1gRSNsJ4EIwkwiL2GgwnqK1NpE50GbyD8dDzgYFXnAy7A0oLuF64ruh0Zo52hdkRK3wLInnj5hHFrQLDMxMwA1PCfdsb5+QV99g4lDG+kwAIQbp2isH9sL5qHH+OCXH14+CXxg5rVyJlha8gnBLXotxPXgVlDirkorI01jLyCot8YVIREDkGz+kp0gmYkrwnQSjrrEw0Ha1EfQHKiMawO8HGs1u0gobfmYNtaC2rlchhqpLcJkSzgZvtC+01x0LEW7eWsbu5qdr3LPAhV1GqRoWDQLg+/1lt3uo/3ltDtkod0Q8TE1tFQ94fHbgrLkQuDaggenZlfyJXuGvSj3Jm2rgRN9V4mi5/NVICovaFdpcu7fbtNjTjRDwO08W+UpcaOempqnR/mzHQKKVrkpCqNfymscabQ0YUFyjgQfCg6V61F+ejbDohk5Hv1UMMWbk0VUlpPjsm8JeWN4nKVq2UHSAxr6bVmKSOThqGrBoRDuYs9l0ghcVgAurUj04VOJb5XQl4ZOVYLy0Bnz1aBtxLIVBJkaD+JI5FhKTm+jnUrHiQmooflCtwwl+b4WprWrJUwzBPO/jaPFG26Zmo8RKrsD8KoT79WaXZ7B18Npav2DH3rRh7ZHrsHDoKVSIOVm/hIYfOpK2dyjF+NYh3JR+LmtOPXxn2UG8Mh7t0rSBYsBagWVUi1/bmeWJK/WP1G3j75QEifsPH9dt6EVOchYWt1Vgo5eHcRW0B9qlWhYX0nvaZwUrjf5uCVd8/2Xd6LprtKoVtJMdM0S9VCuobwJ7aGxJcs61ZYFZ+hjtEDp1irZwlKW0Bh/H9x0diXprd5j+KYU4vnqLcNcrRaI/gJz62/rMaEBZfqG6YPbEsXbeEscgrSwSMDd+kL5Fiub3xmjhQBmV9RpbGbo0mxkGg/k2UqruBS3ED9YwcXAQ1mpm4ITWR60UCtZMbhlt8cFHltnlsizq5wgWYBW13lPablkFkdB8FixyW7ppGsqAd8ypBw0rSm5cq99fMW1hvcruWFa2+hlyI40LedZmEZ/88mH+q0BHWgG0ew9pITEBSXKCW7UQ+9FeS6iVUI1A2Wqx9DE1Ue4zxQkJ284xdAWq9y/N63841dlTBOYhhPQmTvKuiV/ypcGfqOQSNdHal5cGWDZTdDZaciLa+2OxGzzUVazXQqlgi5nj61FK1b6FneTbpqlB6JL1tAKZaSa9bK9CHrgY3sR2Bzqgpj52kml7HIBV1yZ05jsoAi6qKXgBJhGADHQ89BiIp5YKMsoM9VkyQcZATPy9yJKoGHzdBM6vVAs7w7Q02U+Ti3b+vprRWQy8IEGzFwviWPVQdSkxbc+1JTLbjUY7h1rnuNuS8dqtUTVNd0xOG9GNo0c7FT0QVnRZZVVJLIpaalfs5b6XjjnFU31a9ZUGbTcVt8bvHmNN9pvq9tKrBd/ZGP9+jGNVRxe7tBWv96hrfLF8VrpEZCdpq+IWQT67LXjFLPTBLOVERqn2zaHvhZHDe45kwr84nRaPBWDAPefHC/yWZEVOtGvU724Ll7/Ws2F+WaPLACqzjy1J5hBvCMMlmvauOUyZL0yhetOTH92m6ws9bks85Ei9UTVNpmUHMQqoOmPAphvCNlJVz9cERKYdFWMj26QQrnFwJZHgtT9FV0Mhttos5ScK8Tmd8s/JXMXFEcPieYPUReM9gY+vJDnM9raozeVlr4skyIg7mvX2OWuAcwaCPcc+A52dNmz++lkRIXQRD8IXeTo1Optq13rxOrp+qeUelzCLt5mU7jORmVX+TxQTzCajvqV1uPYam2tyMG2ipyWVcQP71SR1vaKGBYzZRK7+8DFH/dbeik6lbd6JJd2zBnF/VWO+2EI1zrUdPihNubVNQkp0EoclwWbpcL+XV/u+ZSlmXzZvL0Mt9Uu3STxr8kRe71lo6yJmXZRVDWgMYBRuLCWBap+DMlT/mbPp7uuVr1dSL5o1LD5lQUvOJBIMcBl4WIZlCVO1OuZilfo7X6JXuEaPfUSvOK1eYV8xkKq7xYrvxiveC8SfU2CPdrAJ3+8awK0BLdEHEmvLIHm8FlXOxmbHSy6xy7S0iM8zsxd1sqg8tM7lWALGnFkbV6z28M+WZTCSharH+LtHS3Vcy6sK5iPmpiB0TieB/QwcI1d/1wOW33ayCCx0nMUOiTWKvPMa512v7ZTfHdb+LqszqsUC6SNtFxrRzb2qs6GZYe8lE5QqR8ySJlyyDAlG5D3tEXnn3BaKFxijjnfBAVffEeNL6u1ICsbRzxJr3tH3rtcADMefrEwKI9dAFzNbDnTofu8YCYSlRU3DWzI4m0p/crN2wVoEVxp9at4zr1Yx6mCU7a/z0goX6xTLFrRd8cEqLpL+oSw0fWBzQB/HpVg9f2XaQWP9fzLVUJM/KpWlG1ZytqyeIc/P+N7y98K0Gyio2ISs+MM1+kfmRIRejPZ2zSoFZZe+dQEeB1F6cvII/VG+ypKUvA252HSFgEocSMj/JwcH4Ll7C/T0+PDNCYkmcEU0IpXYeuKxOT0GDe4W3SH1cn+PAo8fMrjIgTK+6fHVE6nx3RhyULL4WR/dkVm76EO+0acizS6vAwQ9ZAiqeTpbR8X7nQKXvB+VdbXQTp5567bVBAWvxa+3muiT/jPf/yvlt22R9b/+792e2Cd/fMf/we+x5D5z3/8b6ftnFthFP5O4ujIGUDVoYzT7B9eH7lgcPjDC7KGFI94GlN4ohtUxruYoQpAM1bmQybu0+R5QvwXmeJoyNorGPunr8VczJqu3W4Lvo3NRtuUNwooauDPqNIeRrOUpK0EcNzF/mn2shVTfKo97AuVZ5IFL+b0Qv9K5ZowsDbw8CaKQvH+BSeRLnl4AMdF5dGPei0liyUaFkiHxDEIDEw4PBqSG5vs/YJ/e/PjyzZ1detIsM1PcOv0WN1rDaVXAzVEyd7/sNQe0Ta9XsHYzZ++wF69pbuIuohmVKq0X9Eu75II9Od2H3cv7h9Z+09ZgCy7AgfsewzrgUdruXSpvtG2vkcT85Cl46Yr9m4BDQnuN6196bUCpPjdD+OX10PnJo57Tid8cvjry2/ev/ohJlc/DX4+eDY67P/Qe/6qc/PmetiPF70Xh8tHH1bXveXyu+fDd3/tvB8v0mv/Ohi53w0Pwr++7B08Wj5/dX15cpIVRR9UwJJ4j8QM7fECzH30128eF//P6Ei3JwDo7b7rwD8OFmDDjx54I/v56VlIObvdx4MDSBZAIZd2YfoJUJYDOJhKTw1AKrg7+9CNKKlz+H1TRssu0rK30NKOKyFBTFavMKKVEmLKLztCujARQ2ILhqhObzjs9u1Otzce9bt9zHPXKAdTFg+kQjbz4Paza6eRx7bt2HZ/PBj1+k5vPB53xxyC3kZdRlM8y4x1OBsP2IGALn0v1un22GfPoZ+DIeQM8P/0cwQ/h/jyLfscjwDYxif62LO6Nn4jD47DXtF1MGGI70D2WUIHEjp2FxLG7L3aHh5NwBI6XQZhd/BwAv2LFTJA9DE+G2jbNMHBIwxDm6ZiwrCPLGNJI4rRxZ+dbvaNxHCC4OBY4gArPaAv4naBWg8pOs75OSqgsnkfRDRkal4WA2eaJdqcP/q3L3CoemHn2ccYTSXYmQpC7xOhfQ9ULlfa/Bkv1EyRIBR1Wxnwe0shzscXcr69rv8ZWjDeq3Asc3O+kYdgPtzCT5hcT/UJnf2zo42hTfV/t57S7UNH1pNXP1s2J/b1ZToxl8IYwCmeWp0w9LPgwhEaHGTf8j00huKLfEaYjYfudDgm3c641593x87cGY5tezgeEzLqC2sD/kZ6p+zZHrAXFuhNXCjcHoVRWm8rptbRlZvUT5lF0zYZYRTniNprxMPLDxp0sg4wpPilS//AhAe1o6X/MXagxYu/pz1olIWpVQE8cJcJZqLtaFIMjcaXFgf6JHZmJUPlGvUH26O0WvWzM+B7ZHd7/c4YhjL46PbsoTN2uvRj3LNHnfHAalrSbJK/83oGqZ2B0xv0ejCK5tMJg+/3+oPusMvJDnvOoGdZMrLTtjsjZ2QjPIAMhuPRyO6xDxiyRz3b4SX3B+PeuC+9MIts93pde+QMRgzZ7o+GDs5NAN+DSoyGFqPUs/s9xx7hg+3ci6NKyhu0USL9j+vdMC2NOx2Q4wBqMbdHo2HXG85BOp3h0PX6dt67/2c6WF2Han5/JFR/1P/sYn12sf7cLtbLn5KD69G77395fjhe//DoYPTDDz9dxzfpYdwb//b7D4fD9NGz69nT9dXvh09fOL8d9tfO316F4JbFg3D4U+/w9c31k1fJL9+/fDc8fHeVHKbRd9evPrtYn9zFsgcde9CHwbgPzoIzGkkuliFrm4vVhbmm0x3C7DMY290+ekiaj2Ug+tnH+uxjffax/mgfi8wdD+zD7nzesXv9rgOO1nBOevPReNYd9KfOH2uF/c93b3a18v7bOzhD8BTG9pj5A6MeOho95ll0nZ5td3hOZzTq9jt9S3MzxuC6jDs95maMh+NOn/sofOwfUdeHj/Ia8qgPo2qnywqD0dnp0AmAOkSgmn2L5Qx7/ZHTHVmqa9WFsrqDXpeCOMNRJ6M0sIeDvj3kdQCvYjQAtj/WwfloO5fd2fLMoxdG5qbZuDv0nG5/ahOYBGfzgQv+mDvyiD2Yd7pTp0sNk50DIzLwVj9LBt46HJxPuDkezWKwwI0GubCZUVYciINrVvJ+OxPnBWbvF1CfBBHdLCG+2zNMoMXQt18meU5MYKCbkV/89KquoGdE57F7yU9iaivQT/nntxwC6yCglVVMQRjzDYe4s8bVDnGzV2XFKcuMMnzyte/HN8+AuMDG5xUm7D3iaBXPyFN6fUmJDL/E8XzfOrA0dH45kCyWjFp77seJKJvWDBDy3NxB2agtYRTyNn/k/wNfwAcs</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_277cbde407034bf9addfa8e4969bce7e\")) .filter((elt) => !elt.dataset['step1']) )[0]; root.dataset['step1'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>\n",
    "\n",
    "\n",
    "    WARNING:absl:`StandardCheckpointHandler` expects a target tree to be provided for restore. Not doing so is generally UNSAFE unless you know the present topology to be the same one as the checkpoint was saved under.\n",
    "\n",
    "\n",
    "## Restore when checkpoint structures differ\n",
    "\n",
    "The ability to load a checkpoint as a pure nested dictionary can come in handy when you want to load some outdated checkpoints that no longer match with your current model code. Check out this simple example below.\n",
    "\n",
    "This pattern also works if you save the checkpoint as an [`nnx.State`](https://flax.readthedocs.io/en/latest/api_reference/flax.nnx/state.html#flax.nnx.State) instead of a pure dictionary. Check out the [Checkpoint surgery section](https://flax.readthedocs.io/en/latest/guides/surgery.html#checkpoint-surgery) of the [Model Surgery](https://flax.readthedocs.io/en/latest/guides/surgery.html) guide for an example with code. The only difference is you need to reprocess your raw dictionary a bit before calling [`nnx.State.replace_by_pure_dict`](https://github.com/google/flax/blob/764e1732dcd3b8bf178b9ba73ddecf125709b5d7/flax/nnx/statelib.py#L179)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:absl:`StandardCheckpointHandler` expects a target tree to be provided for restore. Not doing so is generally UNSAFE unless you know the present topology to be the same one as the checkpoint was saved under.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<script> (()=>{ if (customElements.get('treescope-container') === undefined) { class TreescopeContainer extends HTMLElement { constructor() { super(); this.attachShadow({mode: \"open\"}); this.defns = {}; this.state = {}; } } customElements.define(\"treescope-container\", TreescopeContainer); } if (customElements.get('treescope-run-here') === undefined) { class RunHere extends HTMLElement { constructor() { super() } connectedCallback() { const run = child => { const fn = new Function(child.textContent); child.textContent = \"\"; fn.call(this); this.remove(); }; const child = this.querySelector(\"script\"); if (child) { run(child); } else { new MutationObserver(()=>{ run(this.querySelector(\"script\")); }).observe(this, {childList: true}); } } } customElements.define(\"treescope-run-here\", RunHere); } })(); </script> <treescope-container class=\"treescope_out_fdd68f8c1cf8422aae61f18e8de4a99e\" style=\"display:block\"></treescope-container> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_fdd68f8c1cf8422aae61f18e8de4a99e\")) .filter((elt) => !elt.dataset.setup) )[0]; root.dataset.setup = 1; const msg = document.createElement(\"span\"); msg.style = \"color: #cccccc; font-family: monospace;\"; msg.textContent = \"(Loading...)\"; root.state.loadingMsg = msg; root.shadowRoot.appendChild(msg); root.state.chain = new Promise((resolve, reject) => { const observer = new IntersectionObserver((entries) => { for (const entry of entries) { if (entry.isIntersecting) { resolve(); observer.disconnect(); return; } } }, {rootMargin: \"1000px\"}); window.setTimeout(() => { observer.observe(root); }, 0); }); root.state.deferring = false; const _insertNode = (node) => { for (let oldScript of node.querySelectorAll(\"script\")) { let newScript = document.createElement(\"script\"); newScript.type = oldScript.type; newScript.textContent = oldScript.textContent; oldScript.parentNode.replaceChild(newScript, oldScript); } if (root.state.loadingMsg) { root.state.loadingMsg.remove(); root.state.loadingMsg = null; } root.shadowRoot.appendChild(node); }; root.defns.insertContent = ((contentNode, compressed) => { if (compressed) { root.state.deferring = true; } if (root.state.deferring) { root.state.chain = (async () => { await root.state.chain; if (compressed) { const encoded = contentNode.textContent; const blob = new Blob([ Uint8Array.from(atob(encoded), (m) => m.codePointAt(0)) ]); const reader = blob.stream().pipeThrough( new DecompressionStream(\"deflate\") ).pipeThrough( new TextDecoderStream(\"utf-8\") ).getReader(); const parts = []; while (true) { const step = await reader.read(); if (step.done) { break; } parts.push(step.value); } const tpl = document.createElement('template'); tpl.innerHTML = parts.join(\"\"); _insertNode(tpl.content); } else { _insertNode(contentNode.content); } })(); } else { _insertNode(contentNode.content); } }); </script></treescope-run-here><div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtHAtX2kz2r8xHz37CKiG8fVTPgvXVVlvF1ra7e9hJMglTwkxMBhD3+N/3ziRAAgG1VavdD08FZu7Mfd+5907s60CMXLKjCZ+QwOQeafucC/Rf5PGACsrZJvKJiwUdkC1kcybyNu5Rd7SJepzxwMMmjA87VJC8+rKJPB9GXBqIvNo6L0YejDLOYNjAZtfxeZ9ZeZO73N8Ml26h6JvhAgDsRy3R2UQ2FQDGBGFiC/Uoy0fjRV3/G+zFr/IBvabMgXXct4ifh6Et5GHLgsG8S2yxiUpmR1LDSL5DqNOBkaJWlfiYwBSYm+wffcgPaEAN6lIBLOK+4BPYPGXCpyygpkRLwtmIr5vXhVCOrydyzPt9Bjh9GAtMn3oCSUFsr2DPc6mJpWgL3BREisknuLeyk83mtndA8oAvEMgiNgvQNhIdGmgOEWeglhNukWxO6/BAaGoeWCMCtT3CJMsNU+4qF/3z32kzh5hZLoFp1nfdrRCDBmS2OGcwmh1yv5tDcRr4BQzJqcSwoKYc9Ihvc7+HmUk0xofZnDIEQJCdm0H5cNFrVC7lYB9qo+wM1ZpLmCM6aHsb6RJkKek+EX2fgdwRcQMyJazTZ5Ky2a2DDrWFpE8ByA838LMAQxbMj1l8qPnksk8C0WC0p9S17+MeyYYyyck9tuYQef2gE4pxK4XHMYrtkI0lXN6dBklFqEjBHccN3betXAys1ZN7yRHiijVEBmDgkSYldeq71iUjKfSMn5EERcCa6eIgeA9eHO2bzUz2bPfADDNj5Dc5kCeYv7LxndeFNAew6ACpDbczyTiTQQIbwCm52s7oGXBdX8yDcAYkgjAYTC1zhnQJZOWaMe8ZcMYw3hlcCN6TgWGTcZHVbO5a2IDVDLbd7OAgu+Nig7g7yZl2iEOt2TQ7xOwSK5dDf5eiGwcewb1NpGvFKunNhx45dp1XLMP3rdQwe6OpcNjGhuGTgbJuFR1f1dZLWNenACbv9YCtGARWL6maGRAcktzhA+LnUuCT4O1hh7A2ufLAIIn1YBJahBX0C05jxSB0eNn2FIL1ewbx4wAb68VqbQJArLaU9BQgOknmD5wB9rP5vOFysxsO5bbGx4dSXNG7QgF3qbUMMrSe24BvpF0SXxLnATmkA3JKMIHr8ic6V5VhbiIqMJwPcnFCrAtOZNBzmvQB2qIBIB2NT95ZQLSDlPY2Nw0CsTomuVemeqWbZnio5ovyVI1OY7DHCS7KlL0rMSzBqaxwHrOF/W5AsAP+z+ZXP5AVTmiQS9MXjeETFKrsYBOt/KtUNcyVX0lectFCImtPQKTUo0Tc9wOpQI9DjkT8FLw0eDi0yhUUoryK6sEiG38YrFP2BLkS81g0GrRt6geizVkYhOZda5kraaWq9KZUVaGfJj/U+CyJkqse9h3IakMylEPf/CQ2CMXeyOhDaGSpAWg6nWa0GZSZgQJBQk2RDvwvUqxYmZmSZOUYg1VQ7KLWqGdwN0Af+kLya6HdcCW8eyNwjPyQGF0oL8LI24OMoaMKCcwELKc4INakKHlFdPmzNW/m4WpVDOjahjzak1yG/pHCRXq4m67Uhjhom3AOgGAn67EtEqfHOE4vwzmzJokyLvroALOwwHnMQLEq3czFhyUSmUP7mI2tWW2LigEiIDHIavK8L+7HyoQCUAwl1h9JShRK9AftedwXmM3tbfi8C4mKHJkGo9ulG1sWk+dYzTdax+y0VTXblt4RS+1CZ9G10GNjcL5M8GKAfpjwTSBlwgesWm0TKiTLJyxiPlmDApVJwAfLvCbOGB3NCec3sWtmoVCGKg0SGpXYa4HAcv2E3kejJMqoQkosyKvATGRaFVPHZR+7DOqetucTm17BJgnHW1eOB0UelpnWEPsMNNAeHxVj7do2NovlFEAPqqQ5xal4GAkpGspHypymlJsqu8d+3vGxRUFtWVQsVy3irCEOTuIQpAN5NbOzFjoNlDcyBKkhFIl5jpa5WP0wBwGaC/n5iXHO6nqRbc7CTVLvUIfg3UmQJ7deXauYndxdaH0acw5LA0VVaoGQqBgVH6ATtwPayhaBnfiSaN8AoEhOVv6RDKZL83dfO8Y7LmPUysmW49F8NPyE0pwWYkqCeelX/SBisEh6wN1ixlJyqHtnMLeZ+H8TtWKoXxmt7lArLge+0WDHR2lHTJX8Mx0J1RCQDYm0zE4m2iC8VJiwJIxBAa2LNgp3AJG42IMM7PYq9v4KXoxhSqiW6H4sgHkIOtJQJEQxBmhzP9bTSxPLYvCHMag1NHNFoCX7geiOBM9KeaKKNqxrs0Tv8seWPSm/dyQ/odTlirwL1juJ6C6yvAuy25h6iKsP2Y1GfzR8H4802+e9rMXNvmxEajLTC7QBdvsEokdOC3iPZFX+J/vY8l0Lqz/Zw75j/ZdZgWwoN7k1CDqECHm1QIZot9VqSW5ackxeFKhJzSeqgdcaMTP7n39ENadJxpno/evPeMePyasSNxobRoG5Ivu8gW9uor7vZmUxtCnnC0Nu26UtA8qtWmXN0jcOjp1Gs6FeR6eNBlefmmdD+H2432jsNZa9mr1Gw+nyd9bRXnN3+LXROP+6+7ZxfNTcbew7V0eH7zsiaB5T4pT333wpvT+qfR20vD79eFw9L779cnT2+XhwcXwtPo7293dXL5zuOW2+0Tv0zWn/7Z518F0/NAr24MjyLt/VOpcXlJ72j9lB59D+JBqfas0Tv9LYP2LdvZr5qd9nq2fVSzPoDgf2vlu4vHL2+LpjvB0erBcPGwXWOKu+9/23xbNV51o/s/TGW7vonNR3hwffS47OR/2zer23V6wND79sfHAcj5x3RxVyZFxXTcP/cCBwwzk9Ohm+wcEoOO0fHX252NsfNj6eekdfrU+FwqpTP69/KQvdfvfxsjGowp7vGyf1xvGw0XOuz1qr/W8tsvflqmTXzOuTytnhqNpvNt5dN797+16ZHp7u7unf+h8rrTqzm+/3DvePew26uj7YK3VYsVNfNT4Pv3wfHvqDNwefdtl3e2/PEasfzG+uW69u7L4dNtc7G5Xj44NW+eBbw+kdVb83TzfE+QE53NhrNo8Oym+cylnhqzkyGgeg08/vCo3TA9wgx7tu4/B674PzTTi15kfnw4ejN80uPa2S/eaX3ea+SXWv43OPgW143/beFK+L3Za9a4vO6B07tPB+cGjrJ72DvZNa02pcfv7sYRG0vvUsC9ONkn29UflEv1/WvJ5f+8C/7raof9AbvD0oty5a5f29ktk8tc9XD13uHVT2g2EVO5e1dfqNtE5c74I1D4+IdeyT/sXlwW6veLHvd1utq2qpdnERDBtAUQ6pa0KRXVFmvSLzoP/Ar4n3Y4t7UBBOXVJdbmqatgRiLfTZf8Ney6+LOuq2TXUBwgYF7A3mwUyUDfsEybtQcMFzLt0XwKI+ghwLIDzILWRjRHYT8BBTgRgeUAcL7muws2dw7Fva0KeCnJMrkZ3uJdPEcK/phRvkbdlMrGsir9oAyzntEd4X2fFd7Nw6n/T4gMwtvVlDJV3XVXoMwRdKkazq/6XjjbVGMlPiZOdzHMHk7WQGvUL7mLoQ2ARHEvgPFdmghGBQrEM0piAzgi3Z2VmNyy66NrzlwlB2iZAKj9uZRKa8iXjXNTtQJ9XXqzKl1cuoVIRvG5VavV4vV8r1KipAWgv8plUJ8roiE+0f3Ugmryhmq28ADvOG15R5/egky6hMweBXmdRNoqQCJsOEAphUi5N4E/dWUfKPkp2nGUqTZ39m509XSEECxHK4xORMMyWzY7v4SmNM/tPCfoY23jR8e68Gs5PBSB5zGMe5XyaFz/HFYmYHLOcj9jEUJCUdZdd11Mwl0cUXr4yLhZXksBxquwM3OZyZa2Qkq7fMEug5UIPiYHuxofyMXaTbwyNrF4pOKilxqTGjX6WObJpeV6Y90xXE2a4MKtsr94yq6gI+t4ImDd3tjCaFm5k6+GQKUig1Jzv0s71omFMRBE6oDnyOGN9ZbDz3tMoKyhZrD2mNcz3fzLL5zI5KcrfvoGtqbWcmN8sbJb1qG1aJVEvlil43N2yzXKoZRt0gpoUxnkGadiMd2Vo6p9/BflRijtIkMin5FeAYxHY5FuVStrKW+9OZWvEtmkoUGqm81jcMUrfLRmkD65W6YWyUDELKRK9umNWKjdeXRfaHcdi04Pd8RHq7D6zd10+WRZtHDQ9hzbcsSIQQDxMqZsMBxIPPNIBgSq/VjqhDLfBWSGvQtBaWRfIPGfVvIb67vy2w7mUKWHT4xR+MyuxAIXAHZc77S275wTsHvzYPP2n8gLHc06de4Km62MhvT6LalFHxpJnUkkfZMku19Op2VSYam7clX99V7qVJAcinCq6JH8zkXjDEgx/NrO+bYCzxJOlvdp+pp1CRIgphgfSrYrFYti2jpMed4jnYubKqZcauAB7tcHgDCpZtPRCSqtuqNcRtVPgUgIoLpoN9k2JS8InHg4JMwQvagLBBAZLvgjcSHc7KWrFYCORj81DtdbFDggJYS6Ed+GaBsULCZLzR3YPtCwpVj63CHw1aFhzZDmHEx+5LDlup0UhWgxFwjM3nFoBipI3DkE7qVUt/VmEoRuUiK46BPFkoKlVqpQcMRhJeWs1vGISeSoE/HIhUTEnRf/S8fGbnhDPywmQueVoobTn59HKmrG0TLPo+CdKkHf7tQWan8rIkHeNqkbxjIE8v9S7xGXH/arE+kvpD8S7SfDj7K9qsxRrK1iovsM9aMWvFcr1eqpSMSq1G1kmtUq0Wy+vrJZ1Uyfqv7bOiysN2Ws31Wr2Gq3a1XqxVSnbJsAkUgvX1ckmvmFa19n/RaV0q1N+l1xpGguXtwjjMX/3WXyjCv3quzy3Letxj9udyq5TO65KiW2UvzCRt+aAEZY4mZ10OXwKpHdXFHFfj8s6tVItX4y9KXUs7TzGQJ6vZy6X6L+gf3t2eeF/8jiVSnK1F5hCHeXpH9mTh0E5pBPwGvUgwGm+kJbOuZ9KJVBPoz1dXpfrWlNSIxnD4OXUiY1ayyIxjII+ZfrwQx/81AruH2/vEpAGg+c26fxO+Fkp9DPArZM57XJCXH2x//L46/iioEsPsdXVCRs/t1ihBXDxTrVSe1b1Rgs7FjhADerpr7Motaaj6BRYic8zIQn7Di6Gn1NCPhqt+QNrzzyjPnBDnfp88Qqfq0QQ/ZmqRzMfzj9pBeXatk8V/NjD9X8zQzP9Fkbl3w+UeHf4xpEUHO/8DrcUPpQ==</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_fdd68f8c1cf8422aae61f18e8de4a99e\")) .filter((elt) => !elt.dataset['step0']) )[0]; root.dataset['step0'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtfet620aS6H89BaJxAtIiaYLiVZQ0ny/xxLt24rEnmclq9ckg0JRggwADgDIVDf/vvMeeBzivcB5lnuRU9QXobjRASnZ2vnNiT8Ymuquqq6uru6qrb8d+cG2l2U1ITvb9IF2G7s2RFcUR2bcC/2R/HicXPpmTJCH+xWgyI6P54aw3cbv90Ww26c0IOSTdwcQb9OfueP/0OF26EfyN9E47bpK4N9fBrxdeHGVuEJHEurU+XgUZaQOcR7CgZOGGU2tjmYA7QTSPAWUOKe25uwhC4G0RRzHFnlpeHMbJkfUHl/6ZWgs3uQyi9izOsnhxZHU7vQFZTNUSlwmpLy6IlqvsLLtZgkASN7ok++fAwjVJssBzw7YbBpcRcBH4fgiU5kGYEeDhEqilkE8aTtOKoaggu2l0O4PmnQs7uoqvqaDKpO9GL1otZiQBglGcNY7msbdKm0B2Fic+SdqJ6wer9Mg6XK4/jST7TZlG8qJNRvTPlBd3ZDnLtZXGYeAXWTWldlKAJEmq60td61EWsmAJOIoiT61lnAZZEEOzuTPgYZVB2sz1Plwm8Sry25xlWpCJ4VkIsEDF9f0gumR65V0h2SCCFmqTaxJlqSjsY+BnV0fQelkbmYOsqYWczcP445F1HaTBDBWnXK1f20HkkzWU3O1262s5i9c71jJet9Mr18eiu/R/WC1aoRZP6EECr7q5Qjlfkxq2vDDwPvhu5t6lxcLYRYleLEiaupdE0h7RozfHj9hYcpwlhKRevCTtZBW1r0gCaamXBMvMorppu8sl8OCiBB7FXkaydgo47sI+3cM/UGyaWYIL68RqNJrWyal1u2dZ8N98FXmIavkkJUkAvfxX8iNIY9zATgMAlpWQbJVEFk19jHQ68yReNNwsngFQy2osKMFFx4t98hpF+ThrdJvNKWBv9qqLeQ5iyA57RUGM1dlNRlLg817lCSJzpI1UIvLR4gVRWg1KvjNbzWFs5yi8ggxnG9cvov8Rnmkxd+KYsRySzHqKyrRwl2/+9OQZaOZUr80lyZ6CMgbRKl6lFLhx7YYr0mJqCJiIJmqIFGduSi5ob2hZ8XyekozxEcwthmodn1hdgWFJ8FCd7pSnMswiZWORMCUSkdMTy6kgInPWCUl0mV1ZbatXIu10VOKCGBOxl6U5RVbkQ6thJu00pyY+XrnZVQfkDjLLiTVLXBTlfG05nB+ppROtQmdFEedn3XNkygEWGLmmdcDJW1VI1oHlcES5dVhhl3WFOfctzDEXNqsrrHffwnp6YVz/z5KWddmyZufmTnsTgevkPU68NIiu3hCg3uDlfSA3dMz/iat9GCxfuaDbifvxVRCxf/Gbk/iTuxRqmVNPMxfs2Vv0T3xeRAMwMndVaDBq9ldB+jyIwDA0aNbf/85UCExVY920HiGCdWw5pN0v8PIKroVmadqcA1BaKXhmSOwhJfYwh8E/FCCMLxvlUg849i9JBo3Cv5bxx8aaAbSsXrOZ6/ZG0uK87ytytE6UMQDzmTy1DE38qDFM/pWV1eEZWRNnOmReayVDjKK0lQXUwl03RLtzhprTipoe5xA5l/+if1i3g9bCPmOd7GktT9bLRq4CqgygGxbqDbqTg4mq5U2flwGiEiU9Yh1ZFcsjTXDMeIBgYawFXFOXybse9o6ppDoCS1OdnBj0GKerqkA+vGJXXxIcZVHbwe7WddZHnKagxXuWIGK0Jp/cWoXSC0kWnyZV1QmhXH0SxYsgAh8jyXU4iBqSCpiqrQ19XASUBWm0a22hIjoKoivNpjCltZ3KsLEBufB3ajaJXGFiS1Ygi9+iI/02S8DjZrZe9d1yIyJMl2Ji3iWXs8aD22RjPbi9xL9mm+Y7o7lB/z5xU3CrLu9XogSB05EIZjU3ANHpOb0h9M8EhujOyBn04Pcl/u6Oevh7VgxSBdqp5fTGhex5ZWw6AbKNKi1A6CTGVgSqVRTm5+FrN4MJUwQ+ELQH/HfTgmkiTcoNJTRyA9U0oA4f/HMsQLiTBWkHB03NQ0vijwDPAc+Cc6EgObn3jNx7IAewOan3MilufOKPZ+/P5VQoJFt3kP83xMsa6F28B97hn6BlOS3J5ys0clNSLcaoH1wGGXWeXyfBwk2wqc4orP2HOf1jt+CnMx+NZn36cz4fzbuE/ux5brfn0Z/+sDfqjenPSX84mvn059gbDPszu8UJksPRyOvRnJk383vspzOaEW9uAwwVk87XWwIpvsrZaI7/o9gu8UZkzDmbzUach7E/H7s8dTKeDOlPbzDr+gP2sz/xJv2cs/loNvQZO/7Mn40Z+xPiu2SQc7aXc+eRMHwLsyhgaTRlGdqkBWYm8+CyNGfxYcT5ISJPAV+McFT3oJHptKVl8SlMkMLIFvjFXIYRbOUDulAI7pxRaEkLuYLQLgxc2vGH0LtqDAZfY9CgaU/3DIoERUFHHFFu2A/8rzmtpznqajRLHYvTRSc4p80+znI9Peu2rOK/85aS4dBUp5zxWTDOm6V5nCr0DkYowAxAlb18wmnrXd7jbr/ivZRICSjF2gTp9+73bPLYlDt6SeLywHfX5jOQQ6PQGwyoPODfpkT5Xo0IInYkETvnsv2tbJa8YZxSU9ZjVZV13tRc8GJqDm34AtoBJjI3iqBZC2JTgVwMgQWYj7esUtuVRWoy05+ruWot86e3HBOxc/fmuV+Wsaz6lmvfq+m6v4umKyRqaoSq7lPfH+u7qrOl0+lNxA2XjxNfbsP4kDdV/R0Bd4pRMM3vuW8/vWtz37XB793k92702g62LdOR+p9zJ8zqzHPZ6yx6b96cx1b3zs3Z/f01Z7dO7t17N+edyRqaU266wm0RDdxU2zaf49/d9zAWWCKh5OEfVQUM2iQ4lVRKabrNvfVFm2VtcyQte+nCZCHj0WG7FH77JI9S7VUi+4yOteey28lBgJ8VsAGmlfhfvM/P733qhrAibmKSUR62adG4TUsK3Ny9CXZWTjr3nsWVeomT2NdukqVPbp4haD4xp3KRJ1wov9F5OQWardeyDs05INjBVoghh+jjv4egQFshBxySYvTvg3GI/w4+DRP+HaK+FLNPOcYcRNc4jweBzl1oJbmzMi/4a8uxvtLikYX3lGNnyYps0cKIXLpZcE3yJcTjYoVTwCzcS3C3V76yBKH7bFK8Bld9c5wOjsd0KG520mUYZA3b1lw9hiQWK49LisVzTE4Dgl4sERbK1PHOFMLn8hiPUl4mdIfERUKWxM3Si3iOq9WrMFTsuCHwp5CdWgcHgW7zeA9PM+CmZaWBTzgPnEvGshQQLMkQAL+n23S4dAC2qQLnkqNDd7kyOk/lYcUQXGMi09iydhNVhb2uK5cHG+9SKgXVijX6B3QsLHkHhQXhjVNYkVJryb8rPQTNGRL9qVnvtZStaMlVqjJ+0rh+plXvzOz7FZM8Y4bBITyvqK6hgeX6Gwce2S2Sg10M+Iun8Tk9DeNekZ3FsLt4K5ZS0I1gS3GPI/9F5AceSRt6JDtg6fgjBT2hqNoGJDGgn/EhgQ7CYi0JkCwYCUzYQrMg58xGP8Y+p54MWB2SuKEtKxsto7NcpVcCgTJqG2NPZZK6xy5YzzfN8GqecVR3HaT2+blq+ATwicWh0g/B8oKOQ7a21COx++7BrQF8c6Qmk8iHxHfm2Tgv+Piu5TI8E9VtOFbbVBbuTML1qnq+d2kP1nj6ollpQaTMHkW8Q43sxsJNPxDfildZ074XmxdhHH9YLUvcivUb65tvrK84bnAZxQlOEOloWdM61XyVq8NUNV3N0gx8NNp3cxVkvF3QtWr7XJsuCk5V1KqZY4nDVfQhij9GCnsVXoOEJxdWZZd2kT12QZPo0bszd9sOopj77On2rpPTvE8PKOvVnZqNFamxvmurbW2z+h5S0V6bqomIXNzxH09tbVIRh6RDkiROGvaPjBd57Le5HTHu7OK7AFgB7+MgEnMPZYPpY2jkt0vilVZpL8D5c29+jLIg/Ilt+G74BGN/dHtyy3IpmJCdZL/o9vAArM7ND7OUJNd08w7fB0uSlFA8kdVogI+bBCTNNzKL9uLpZ93zTiAhvsHiQQG7ZevDt2K/cpMPuO3+xJL47fyyIsnNW/BnvSxOHodhw9a3bsuiZ5VryMsSYipEQuwwWmGqCgFIJyGL+Jo0miZVLkuo4wcpVCJC30NvzJZ1u8n3CkM10uxxBBMHZPB54i6ItAm8gnjMfsjtJ1wZ887u2SoI/cd8n/nz4HKVaI3v0WiJqPU2VVEZvNiVusqirJp35E84T/OYBbEx0INf8m5Xuqv+iZuSYb8AkhJLsM9YqEgBpWkyJLVbr8A06ZS1DBkHTyBg0MCn29A5vJQow65F95VAizQZ8sYAeWOETEOwAb4BXMuQcdQoWoHiaTtBJC83WMNI+JokuA2kQFCSNUmuyHPqY+P6wwvJAVakWgU03SvGKezGtL2K7VZyk4K1nLNjBpKlpABsTNEPPUhaUt5FoREOoq1k2amESqICM7vCuQmOq98y+7CK0tVyGScZuEE+tfzN8nZ1qncX6CyphbJzIppWNmWh5ZKLQ5zk8q3wmOCtkgRGazUxJUs6ienKsxgtlJTv1S1UtiOCDzd6UlPbY4YFMLvJyy9i/zk/yOqB+M539lP+lcSNXk+1xlmcueHTOEy1esfhX/GgFK2nc15ksOqAVEvTOE0ApXob99GlAAA4BbAUOuNxTNyRCzkd8J7Ap6I/qYslwAqOmMgK10SuW/H7IZA8oCKGsnCnf6Qco8hrzqjleIrC5ZJ7E3/UJAea+x0JLq+ykuhudhXdzV1Ed/MJorupFx2vXPF7i+iKqkuyQ8RmhSpyk+NhXO0tjsA/UO8WC73dVMtHG6x3EJKGIUmqXPgZlVME3se5dAZoT+fac2HmlspLdw0/9lYL6HgdLyFuRr4NCX41bAZq58eo6GeHHkTEvd+Fah5YPTweIXYfKuBXVLI5PG0PM7wk1KcUF9dJyTpTeOVU+Soq5Dbsnk9ZNAwSfMc1PZvy73xjthInLmL24H6iecMmVF0jaRVEWM43xQZ4OeRct3Rl3J9nROE8q94r3+cPBZbgF9JxAGlzf2l3ON/dr+8ML60vsz0x9ATI41lqKjHPVDcm5IjuugaRZpYYNjfUCZNveS2j1Ax8QzvdcowCwS33uHlbF455/Z47kHQD/CtBsuZ4U/HHxHcLS20VImwVQmlZ3Y4zaE53rY7CEyYe4AG+R8W5vK3BcL5ehoeEDKoTRIYmvIuescNHC/mgzrZKdTuDHVqkpoXbWB3awsgp+9oShNkznO4A5/2FfoJSCuvSARjmlNoIrJy3yEkcnMgzBHkgflg/UE91DvkUorAnkjuCGsjGpq62p+KmgL+ph+fDWGH6LTxWHRIYnRM8tHQrrZ9wr4DB5qmV4MpPaYIA+NRxPsvldV5WOhFoPFG8YT4AWH+k6zfWkfXVV0V2BT3DTnY5gqDZljvscN8rrdspOqrooPknc6b8azfyyNN4FWWy7t3XqeIekdAt8G8OpMbNHRxMLfwcCiYcIhW2UqsVh01V32JEkPk4PZEcNVwuLq3Dap+qbMrVUKupyU7lvb2Vd+iaCnc6LZQZUGlUCK1Zj1+x8Kx9zsDf+lC9dLzRhltNd05ycRj3I2illcqS7Ya1V961oYwD62q1vc8kis5/alQWh6iDk2JOU6mvldq6rtdWFOda1dV1na4qH+sterqu09JKHV0bdXRdrWMoJNRQs5SadchG9dxZXRT3da0r5bpOKfcqS6gy1uo/hiG8EwYR+SuflDjTGsA0S+IPpGIxv4ryU3eJwOkvKzchW6H/Laaulr3ABV77NzWxe7WGjVfWtOeELaTnR8Xa6JK1DKPfNpg896Gsg8659cc/opuKexNqMKRxtQqlyqSa7KhTaUedL3b0ix2tsaOnn8+O7u1mPJ0K4+l8MZ6/a+N5+hmMJ/1bDoYVN6qQTIlTNCLyUfxW9ytJGWjQTUGOJl/KFixURdoKSneMkuGWk+2RMTlyV1qk1SOCuSTY1oTXuKyF9x3cVB1zoPNk6UYiGnygARhhyPCWD2dqOGEvY91UYKkCwX3OeH0N4vOfmHoqd+Qixl/co3QOkHvqcHuqmCgpui1hGW7DkWOZ9V6YODiXrsJMinl/tsiKWBxmRIr4P3TQncIpFudNDYlTetM7hm8UpI0aRUFtx3x+31TpLoVAjd8U0sdx/pRNltrtcsXrFpdyGMqstX2Y50G92F+Fq1SFp7c/5Tj0S8bT61d8fi3oTUvHXePE167xkhEfcb5L2/SLe8iQwIFhzUd0FMzn/YMhnRaLRepwaVBrefz97VUEep4sQvg0qsjN/VXkZicV2eqv6joiI9QqSbmG91MSBfH3oyTinjU9PtqyjGHOFmenCFqe1y5QFldiYjxfuieo2IKFocvIfwoTTr9yGdAPru2mepdiEFGq0rqcvqLCCr47fUpZQWMrfmKBryBNL/DMVyNt6WJUe2oEzVcid4DFq4Gf05uB6Wxe3A1cAZ0lbpTiZvMfkuCSBQCyeAljwLyKPhiu10m8JEl207CDhXtJ2glB7Q+iS7zihe65ASH5dnMHAu22uIC0/WscL5CAsyMi3jbWphchp+CZUMz+cm0LcbPmUEX9znNDr3HtJg2tXPSaH9zKy8Sb5VqcCpQp5S2xGykGXkFLXE9LDxKgzMA9sIWyMPydG0kGV+RE70lG4ZhYtcuLz7rQ+R2/L6GsvNYPbksD/4Ze4NgZkAWOs1BfUeEKen+JlxK59U7kSrsYwvClOyOhvLmDryrR9Nfa0QduW3D9fQH9gsLQkBpdOg/xU1k/pylCg/Dyq7fYkVD6SymeJkOxm5Kf0Nu2Ea7bcaAG0p3JRizUYXb1kTQU2VQ0eG2bQ5dlG93OEAfXciM2UeNYhtolmuI4yua33zNE66Odqq4fNU2SkHuEuLLZKLJQVsfqTshnLVL/09uLtxQj5HS7X4O+PbgNUP+o+pnx+Hgi1XYbJ7rrWkQoDcxhFPQpG+TFJIJmmFhB2Md4AzvKTBoRLKOicy3PZ7Qlk6Xm5z2M7S6SMze/+V6q/2GVUkbNJM6APA6b7UnXJ5e2kbZhXOYaleCwbywmUezHzhpn0tS2dX/Nv7MGlx3rWhU2KKcqkx20c0/1oEFVX95bJQr0rb1LAt1BhUrQs2L0rwYSWlAHA2O/WBuysb2rIRXVzZWyHVOtRCVWtFJTZbn3F5TvO0QYKEjGmMHcUqAjqR4tPGmFSZT2plm1A5AGXp7Ea5JWe/CfNEcoCuh4oZumL4M064DHAp5uNI9RlPwZhtxzukt4iIOxWw5rt4nyWRDbVfyWM6UpvraSILG+S7XxWRJbusVZK4vTaLz7z+jBbdFHNmfvtA08+ABGibWqQunLGVKvZMg8imrZ7DENW8sVW3HKUuIAbI9Ydba7VsXtaABpRpby2oZZFEycDGWb1Oxzy9bkxPTmnnJiyLmc6LsotpZZLSYOUCUmkV0pJg6giylPLs+w0F/td70rvF0l5QaisBB18mVEy7LzMHAKOU9hvMGHSuhzFeQ6Uxz2kiwAoANVvSQZSypCH5puVQPuqSvV1duW+RUTOiWle0txoYr4v97kMPZ8ixc34EBEYLgUutEyCUXvV/dBLg+6GMUJgcJnHHI5xWL8ESkdupHOzv2PBTsQZ5fOruNV3zDU4kF+vNSmM6Bn6h28hEZeIVBr8MOSm3GF/7zsXViPKQl54FSJ5xqV3xsun5dBd6o8EOw4DBTIxoFAymY93XYMObST2z1DFu/ethvdlDNFrQwHqmQSJHucZUkwA2PesGkLt+Sm1QJx8xhf2vp8AT5O0Wi5NZD6Sb2AEqP5f8T4eFZbJ8KzCwGY8+0DOp+nJ5PncbMcx8ifaarsYGUnbCeBCMJMIt/DQIX1FKn2VOdBM+SfjwccjMo84H1YGtDdwnXl6YdGaOdoXZkS98DyR5w+Yxxa0KxyMXMANTwnXbO+fkXfdQPDoY10GADCjVM01o/tBXboCT7pFUSXT8MAmHmjHAqWlnwimBa9EeJ6cCso8alKOydNYy8gqHfGFSERA5B8/oqdILmLK8J0Eo66xMNBOnSOoE2gcq4N8HKs1TxFQmnLB7WxFtTP5TLUSG0RJlvCyfGF9p0WomMp2uVb29jV/HyVexaoaNAgRdOiWRh8b7S7nQH6X06nRxbaJRGfUkNL1RMevy0pSyEEri14cMq7ku/ZM+xFuTfprho40XeVKHo+X4Wi8oJ2nSYX89ttesyJ5gi4nWerPCVu1FNT8+yoeLlDQNEqt0Rh9Et5kCOLlyYsSC6Q4EPBoXI9Kk7P5lg0o8CjnwqmeFWyjMpyClz2LSFvDO+z1C07SHpAQ79tSxGJPBzVLTiUwl3sQUwagcsLwKUViT58KvGtCvrS0KlKUB46E74atI1YvoIgU+NBHIkcSynobbRT6WiYgBq6L3TLUFrsa2Fau1qCmSGY/zyJF2+5Z2o+RqjsDsDbTvzXa3Z9Bl8Pp6mNj0Hkxx87PrmGGQYtlQIpl/NXwOBrV8rmHr0Yx3okF4Wf24pT3/9Z5gCP/ferNFuwGKBWUCXV6hd3vDR9vf4LnfbRR0qSlJ3nb+g+tCIHGUuru0rQ0auD2Ar6Q60STetr6UmNk9IFJ5+2pGu+AvNONN1VFtsVzUTXLFEP5RrKm9AeGluyqlNtWXCGPkYLlC6ukj0sZQmN8ffRzbwrSW/1HsM3pRA/UC8a5mq1QPRXmNt410gIDSjTV0of3FYo3sZf4hCkhUVCPq0vlW+xsvmtMVoIwLuik8ZWji5ZI9N4IFsrreJS3EL8YAWXAw9VpW5Kk8jqoIVayZrBLb9ALvTZOrNEnt3mBMkCtL7Oe0rLpV4Sh+ETxSe7pUbXVAI+Z0g5aFkzcuVeB/iQq41XLLlRZm/0MuSJNC3nRZTFPwXkY+PWgA40w9j7ACkRcUGJCoIb9dB7WZ6LeJVSzUCZ6jE0cfkR7jMFyckbTjG0xSr3t5ZVfPysjGkC03ACOp+Osm7JH+ulgd84ItINktosrgqw6jLo/DTkxbV2TWK++Siv2S6FUkFXs8fWohUvfct0k26apQeiK9bQSmVkmveyvQh64GN7Edgc6oKY+eZJpexqAddcmdOc7qAIuqil4AS4RgAx1PPQYyK+WCjLKTPVZMkHOQEz8nciSqBh83QTOr1SrOgO0NNlPk6trvXNN4rIZOADDZhNvSSO1QmiJi2+9cFW7rvVYPjsWJs57rZ0rFZLVF3THcPkzcimkYOdij6oKrqqsopENhUt9XPeUt+JyXlNU/2cN1UOLbfVd4bZvMYb7bf1bSXWiz+xsX7+lMYqDy93aKuf79BWxeK4XXkEZCfzFTOPQLdeO5qYnQzMVkZonG6bDX0jjhrc05IK/LI5LZ+KQYD7G8eLwiqyQqf6jaoX1+UbYOu5MN/skQdAVctjP8UM4h9hsFzTxi33IeuVKV13Yvqzm7Gy1BezzEeK1BNV22RScRCrhKa/C2C+IWQnXf14RUho0lUxPrphBuWWA1s+CTP3Z5xiMNxmh6UUXCE2v17+GZm7oDh6SLR4i7rktDfx7YUin9HW3r2p9fRlmZQBcV+7xi6fGoDVQLiXwHe445Q9v59ORlQITfWD0GWOTq3+ttq1T6y+rn9KqccV7OJtNqXrbFR2lc8D9QSj6ahfZT2OrfbWihxsq8hpVUWC6E4VaW+viGExUyax+xy4/ON+Sy/lSeWtHsmlHdOjuD/LcT8M4VqPNB1+qI15DU1CCrQSx2XBZqmwv+nLPZ+zNNNctmgvw3W1SzdNg2tyxB5w2ShrYqZdFHUNaAxglC6sZYGqHyLyjD/b8/muq1VvF5IvGjVsfmXBCw4kUgxwebhYBmWJU/V6pvIVertfole6Rk+9BK98bV4pn7GQ6bvFqi/GK9+LRB+UYO82cOOPd02AluCWiCPpoSXQHG51tZOx+cGie+wirTzC43ju0q6CKk7v1IItaMSRtbnd7YwGZFEJK3msQYS3d7TVmXNpXcF81MQMjM7xPKSHge1rptKVsPWnjQwSqzxHoUNirfKZud3rDOyd4rvbwtdVdV5lWCBtpOVaO7KxV3c2LD/kpXSCWv2QQaqUQ4ap2IC8py06/wXNQukac8z5NizNxXfU+KpaC7Kyc8ST9Lr35L3LJTDj4RcLg/LYBWCqmS9nOnSfF1giUVlx08CGLN5V0q/dvF2CFsGV9qCO52IW6zh1cMr2d49E8sU65aIVfXdMgOp0STcIG10fmAX4/agEq++/TCt4rOdfrhLC8KtaUbVlKW/L8h3+/IzvLX8sQPOJjspJzI8zXKd/ZEpE6M10b9OkXlh2FVAX4E0cZ9/HPmk0O1dxmsFscx6lHRGAEjcyws/p8SPwnINldnr8KEsIST0wAe1kFbWvSEJOj3GDu0V3WJ3sz+PQx9c8LiKgvH96TOV0ekwXliz0HE72vSvifYA67BtxLrL48jJE1EcUSSVPb/u4cGczmAXvn34TZlM5247ijGbap+/ddYcKwQLmAUIhg3P0qAAUIPzy+Ea/1aygav3zH//V7XSt//O/8e+zf/7jf9FHfv/5j/+Gf8+tX0kSH/W3FCiyxT9cQnItKTRZQ4pP/H01D8+IgxL6Fx4qFShGbT5k4s5Pnica9CJXRQ1Ze1lj//SNsO5MGTqdjuDbqAhUS3gzg+qHgUe7waPYy0jWTgHHXeyf5s9lsa5E9ZF9oTpO83DInD4RUKuuUwbWAR7exnEkntTgJLIlDzjgSKs8JNKwM7JYoquCdEiSgMDAKcTDJoX7yl5E+Le3P3zfoZPnBhLs8DPhOj1Wd7upjBNADVHyN0UstY91TO9hMHaLxzRwnNjSAUVdRDMqVdqvaZf3aQz6c7uP+yH3j6z9Zyzkll+qAzMGDBTCHNly6eJ/s2N9h07rI5aO27jYSwg0yLjfsval9w+Q4mPjn5OTHJQ+sYCQvPdhhvacAaXz52+fPP4zw5PuT4Cs2323C/84MA3ZL47NQsrZ7T6eGEBsgIBc2tPoJ0BZ3SOrj6n0uACkQj/eB22HX/3NOfy+UWhhinbaSCSrNxBRjkSdiruKsCSwo1gOjhj7C3ed/+aBTvhmM6z9/FroHCS/HTpPEU8jIxdnkyHbkX9I32x1Dvvss+/Qz+EIcob4f/o5hp8jfH2WfU7GANzFZ/LY07Zd/D6cwF8Oe8nWwYQRvsU4YAk9SOh1DyFhwt6M7ePZACyhd8gguj08HUD/YoUMEX2CT/d1uzTBwTMEoy5NxYTRAFnGksYU4xB/9g7zbyQ26efgWOIQKz2kr9IeArU+UnSc83NUBGX3PIhoxLSqKgjNtEW0Gn94b1/gUJVBXd3HIEkt2JkKQi/0oKoOSlMoYvGUFmqbSBDKt62M8+1s/GdkwcilwrHMzflGHkz4wAE/wUyUTFPJjmn2t85o/d16RrfWHFlPX/9odTmxby6zqbkUxgAaK+qRwSDGJt5HFi4U7VuBj45CcpGPbZNedzCf+T0y6B32uyNvMvcOe8PZbDQjnu+6Lmdvm/+QOwqNs24H714X/we9oq+xnPCBqWni/a5Me+PhaOgO5oORM+z35r3ZnDiH89H4sNfte/5gaGT6/w+fyurXeVVtGNXGDnWsBkPqWUHK5JD7VuPxOUqU+lfO8IuD9cXB+uJgcccovE5fvP4YPj54Ofx5/Kp3EL7wyPX4u3SUDG+SX14cDP/0l+9eLw5fPR0d/Pg4vHz0t+c/ra6dD92Xs8kv7uDPk78++7N3Pfz1pyfPRulV7/Gj/3h58/L15X0dNP3/FQ6bgw5by2KeW7/Wc3PKnptzJ8/t07zAz+Pz0bGsOxlPJuPucDIaHqJPlfuAhqxtLuGo1x91B12nOxz0Bv2eU3IRDTS/eIy/V4+xZW0rxPniln4ut7TvDZ3DEXTQ3qw/HJIxjI+DgXM4Hve6ZEDGv62HZ1Ew4u/i6RmyvHiBceiLOv+I5gF46C5TzKQOo6FNNBp/MDqN/886l7USqdam39gX5ROZMzymdzjojscTq4XGpz/uj4eHgwH76PZ6vWGP5QydSW8w7MGIKSLeZwgyHA4H3S7IFSgdOqPR4QTHe/wYT/p9Z3RIkQf9IZTR7UvIADLuH47GE7Qo3OxNIIMVNhoOxvSjsGGWWvLQ6QL1EQNxepNRj9omJDsadUd9jtwb9SZgKcbnMJazoLsyVauQvqFnf7KXzW6geOHT6+8Kx3A0mZHR/HDWm7jd/mg2m/RmhByS7mDiDfpzd0zdop2nsjLw1imkQnnbaHQ+5ZOB2EvA/zdOB4THjrLiQBxc89H3O7k4LzB7v4T6NIzp0q/47niYQIuhL1lMi5yEwDjrkb8G2VVDQc+JzhP3kp8r09bTnvHP5xwC6yCglTUZQRjzDUdS88bVjqSyNzLFmbGcMnzylbwnNy+AuMDGy+Kn7HXVeJV45Bm9jKFChn9Ac7JvHVgaOr/qRBZLTq0zD5JUlE1rBghFbjE92qgtYRTyttnQ/wUgSRqq</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_fdd68f8c1cf8422aae61f18e8de4a99e\")) .filter((elt) => !elt.dataset['step1']) )[0]; root.dataset['step1'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "class ModifiedTwoLayerMLP(nnx.Module):\n",
    "  \"\"\"A modified version of TwoLayerMLP, which requires bias arrays.\"\"\"\n",
    "  def __init__(self, dim, rngs: nnx.Rngs):\n",
    "    self.linear1 = nnx.Linear(dim, dim, rngs=rngs, use_bias=True)  # We need bias now!\n",
    "    self.linear2 = nnx.Linear(dim, dim, rngs=rngs, use_bias=True)  # We need bias now!\n",
    "\n",
    "  def __call__(self, x):\n",
    "    x = self.linear1(x)\n",
    "    return self.linear2(x)\n",
    "\n",
    "# Accommodate your old checkpoint to the new code.\n",
    "restored_pure_dict = checkpointer.restore(ckpt_dir / 'pure_dict')\n",
    "restored_pure_dict['linear1']['bias'] = jnp.zeros((4,))\n",
    "restored_pure_dict['linear2']['bias'] = jnp.zeros((4,))\n",
    "\n",
    "# Same restore code as above.\n",
    "abstract_model = nnx.eval_shape(lambda: ModifiedTwoLayerMLP(4, rngs=nnx.Rngs(0)))\n",
    "graphdef, abstract_state = nnx.split(abstract_model)\n",
    "nnx.replace_by_pure_dict(abstract_state, restored_pure_dict)\n",
    "model = nnx.merge(graphdef, abstract_state)\n",
    "assert model(x).shape == (3, 4)  # The new model works!\n",
    "\n",
    "nnx.display(model.linear1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "    WARNING:absl:`StandardCheckpointHandler` expects a target tree to be provided for restore. Not doing so is generally UNSAFE unless you know the present topology to be the same one as the checkpoint was saved under.\n",
    "\n",
    "\n",
    "\n",
    "<script> (()=>{ if (customElements.get('treescope-container') === undefined) { class TreescopeContainer extends HTMLElement { constructor() { super(); this.attachShadow({mode: \"open\"}); this.defns = {}; this.state = {}; } } customElements.define(\"treescope-container\", TreescopeContainer); } if (customElements.get('treescope-run-here') === undefined) { class RunHere extends HTMLElement { constructor() { super() } connectedCallback() { const run = child => { const fn = new Function(child.textContent); child.textContent = \"\"; fn.call(this); this.remove(); }; const child = this.querySelector(\"script\"); if (child) { run(child); } else { new MutationObserver(()=>{ run(this.querySelector(\"script\")); }).observe(this, {childList: true}); } } } customElements.define(\"treescope-run-here\", RunHere); } })(); </script> <treescope-container class=\"treescope_out_9572198988a54a0b9731fab9e0c97e51\" ></treescope-container> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_9572198988a54a0b9731fab9e0c97e51\")) .filter((elt) => !elt.dataset.setup) )[0]; root.dataset.setup = 1; const msg = document.createElement(\"span\"); msg.style = \"color: #cccccc; font-family: monospace;\"; msg.textContent = \"(Loading...)\"; root.state.loadingMsg = msg; root.shadowRoot.appendChild(msg); root.state.chain = new Promise((resolve, reject) => { const observer = new IntersectionObserver((entries) => { for (const entry of entries) { if (entry.isIntersecting) { resolve(); observer.disconnect(); return; } } }, {rootMargin: \"1000px\"}); window.setTimeout(() => { observer.observe(root); }, 0); }); root.state.deferring = false; const _insertNode = (node) => { for (let oldScript of node.querySelectorAll(\"script\")) { let newScript = document.createElement(\"script\"); newScript.type = oldScript.type; newScript.textContent = oldScript.textContent; oldScript.parentNode.replaceChild(newScript, oldScript); } if (root.state.loadingMsg) { root.state.loadingMsg.remove(); root.state.loadingMsg = null; } root.shadowRoot.appendChild(node); }; root.defns.insertContent = ((contentNode, compressed) => { if (compressed) { root.state.deferring = true; } if (root.state.deferring) { root.state.chain = (async () => { await root.state.chain; if (compressed) { const encoded = contentNode.textContent; const blob = new Blob([ Uint8Array.from(atob(encoded), (m) => m.codePointAt(0)) ]); const reader = blob.stream().pipeThrough( new DecompressionStream(\"deflate\") ).pipeThrough( new TextDecoderStream(\"utf-8\") ).getReader(); const parts = []; while (true) { const step = await reader.read(); if (step.done) { break; } parts.push(step.value); } const tpl = document.createElement('template'); tpl.innerHTML = parts.join(\"\"); _insertNode(tpl.content); } else { _insertNode(contentNode.content); } })(); } else { _insertNode(contentNode.content); } }); </script></treescope-run-here><div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtPA1X2sq2f2UOXesIVwmET8HqeqCAttVWsbXtu3dxJ8kkGQmTmAwg3uV/f3sm4SMQUE/V0/ZdXArM7Jn9vWfvPWnfBnzikAOF+4QEuuuRnu+6HP0HeW5AOXVZHfnEwZyOyB4yXcazJh5QZ1JHA5e5gYd1GB/blJOs/FJHng8jDg14Vm6d5RMPRpnLYFjDet/y3SEzsrrruH49XLqHom+aAwCwHzW4XUcm5QDGOGF8D3nYMCizsg4xeR0VdFsgYSRrE2rZMKIqZbEN45gCzbNl0YfsiAZUow7lQDkecncGm6WM+5QFVM8G9I6EsxG5929zoXjezsST9YcMcPowFug+9TgS/O1vYc9zqI6FxHKuzong3id4sHWQTmf2D0CggC/gyCAmC9A+4jYNFIvwC5D2mWuQdEax3YArch5YIxz1PMIEyw1d7CoW/e+/kmaOMTMcAtNs6Dh7IQYFyOy6LoPR9Nj1+xm0SIN7BUNiKjbMqS4GPeKbrj/ATCcKc8fpjNQvIEivzKBsuOgtKhYysA81UXqJasUhzOI22t9HeQGykXSf8KHPQO6IOAGZE2YPmaBseevApiYX9EkA8eEeftZgSINVMcMdKz65GZKANxgdSHW1fTwg6VAmGbHH3goibxjYoRj3EnicotgP2djA5eNpEFSEiuSuZTmhV/ak54C1emIvMUIcvoPICAw80qSgTn5X+mQihJ7yU4KgCFjRHRwEH8A5o33TqdmevQGYYWqK/D4D8gTzlzZ+8DaX5AAGHSG54X4qHj5SiGMNOCW3+6l8ClzX56sgLgMSQRgMpjY5Q7IE0mLNlPcUOGMYxjSXc3cgAkOduTytmK5jYA1WM9i2buMgfeBgjTgH8ZleiEOuqes20fvEyGTQP4TopoGHu14d5RW1TAaroUeM3WUly/B9LzF63isyyvWwpvlkJK1bBr03ld0CzufnALo7GABbCxBYvoRqlkBwSLLtjoifSYCPg/fGNmE9cuuBQRLj2SS0DivoF5zGWIDIw8s05xBsONCIvwhQ21XLlRkAMXpC0nOA6IBYPUdG2E9ns5rj6v1wCAxZA+TEDxWnercocB1qbIIMrech4Hthl8QXxHlADrFBTjEmcFX8RMelNMw6ohzD+SAWx8S65qAFPSdJH6ANGgDSyfRAXQZEB0hqr17XCMTqBcm90eUr2TTDQzWrilM1OnzBHme4KJP2LsWwAae0wlXMBvb7AcEW+D9bXf1MVjijQSxNXjSFj1Eos4M62vpnoazpW38nefFFa4msvAKRQo8C8dAPhAI9F3Ik4ifgpcHzoZWuIBFlZVQP1tn482Cds8fJLV/FotCgZ1I/4D2XhUFo1bU2uZJSKAtvSlQV+mHyQ40vkyi4GmDfgqw2JEM69P0PYoNQ7E20IYRGlhiA5tNJRptCqSUoECSUCsnA/yRqyUgtVRpbpxisgmIHdScDzXUC9HHIBb8GOgxXwrs3AcfIjonWh6ohjLwDyBhsOLohp2ccllMcEGNWa7whefGzt2rm4WpZDOSVmjja41yG/pHARXK4m69Uxjjo6XAOgGBn67HJY6fHNE5vwrm0Jo5yUfTRAWZgjrOYgWJluplZHBZIRA7tYza1ZrktUgNEQGKQ1WTdIX8aKzMKQDGUGH/EKZEo0R904Lk+x2xlb813+5CoiJF5MHpYugvLFuQ5VfO9Yut2TxapPeEdC6ld6Cx5JfTYBThfJHgLgH6Y8M0gRcIHrBo9HSokwycsYj5egwKVccBny7xmzhgdzTHn17Gjp6FQhioNEhqZ2CsBx2L9jN4XoyTKqEJKDMirwExEWrWgjpshdhjUPT3PJya9hU1ijrcrHQ+KPCwyrTH2GWigNz0qpto1TayrxQRAD6qkFcXJeBgJKRrKRsqcp5R1md1jP2v52KCgtrRaLBvE2kEu+IhFUB6oq+j2TugzUN2ICCSHUCTlFVJWQvWzCT6m8uzMMpcVvc4wl+FmeXeoQHDtOMirm25eKel25jG0vo4th3WBpCqxOoiVi5IP0Iljo3+gtArsLC6J9g0AimRE2R/JYL40+/i1U7zTGkaunG05Hc1Gw68ozXkVJiWYFU41DCIGVTIA7tYzlpBAPTl9ecjE/xMrFEP9ilD1iEJxM/C9Aju+SC9iruQfaUfIbgAQ62APEqOHi8uni349hrCklDhiTYk1MM9BRxKKiNE4QM/1F1ptSWJZD/48qt5BSw15Jd6mQ48keFnKM1X0YB2QwH942avy+0jyY0rdrMjHYH2UiB4jy8cge4ip57iREE1i9EfD9/FEMX13kDZcfSj6g4pIwAJlhJ0hCdKZjBK4A5KWaZloL4t3JSzKRGv5kWVZagtlUGbWzA9sQrjo+JMxOux2u4KbrhgT/Xs5qfhE9tW6E6an//0/USmok2mC+PSycLERx8QNhhONjaOQWRLt18DX62joO2lRo9TFfG7smmZhT4MqqFLaMfK1zqnVaDbk6+S80XDlp+bFGP4etxuNVmPTqzloNKy++944aTUPx98ajctvh+8apyfNw0bbuj05/mDzoHlKiVVsH30tfDipfBt1vSH9dFq+VN99Pbn4cjq6Or3jnybt9uH2ldW/pM2jvE2PzofvWkbnOn+s5czRieHdvK/YN1eUng9PWcc+Nj/zxudK88wvNdonrN+q6J+HQ7Z9Ub7Rg/54ZLad3M2t1XJ3Le3duLOrHjdyrHFR/uD779SLbesuf2HkG+9M1TqrHo471wUr706GF9XqoKVWxsdfax8tyyOX/UmJnGh3ZV3zP3Y4bljnJ2fjIxxMgvPhycnXq1Z73Ph07p18Mz7ncttW9bL6tcjz5vtPN41RGfb80DirNk7HjYF1d9HdHn7vktbX24JZ0e/OShfHk/Kw2Xh/17z22l6RHp8ftvLfh59K3Sozmx9ax+3TQYNu745aBZupdnVb+zL+ej0+9kdHnc+H7NpstSy+/VH/7jjVcu3w3bi5a9dKp6edbrHzvWENTsrXzfMav+yQ41qr2TzpFI+s0kXumz7RGh3Q6Zf3ucZ5BzfI6aHTOL5rfbS+c6vS/GR9/Hhy1OzT8zJpN78eNts6zXu273oMbMP73jpS79R+1zw0uT15z44N3A6OzfzZoNM6qzSNxs2XLx7mQff7wDAwrRXMu1rpM72+qXgDv/LR/XbYpX5nMHrXKXavusV2q6A3z83L7WPH9TqldjAuY+umsku/k+6Z412x5vEJMU59Mry66RwO1Ku23+92b8uFytVVMG4ARRkkb+94ekua9ZbIUP4Nf2bejw3Xgzpt7pLyzlFRlA0QO6HP/gv22nyLY8tLMFmch30D2BvMg+koHZbv8StKcMFLV7gvgEXlvRgLIDyILUS/QhT5eIwpRwyPqIW56yuws6e52DeUsU85uSS3PD3fSyRw4V7zezDIqNKphWaGuAEDLJd0QNwhT0+vSFfW+WTgjsjK0vsdVMjn8zJxheALRUJatuWS8S50LFJz4kRDchrBxKVhCr1BbUwdCGzcRQL4DxnZILlnUENDNKYgM4IN0XDZXpRddJv3wD2eaN4gGR73U7Ecto7cvqPbUMFUd8si2cwXUUGFb7VSpVqtFkvFahnlIOEEfpPyd3GLkIr2jy4K4zcHy1UxAId5w1vKvGF0kqVkpqC5t6nETaKkAibDhAKYlIvjeGPXSVFajuINoSVK42d/6uBPhwtBAsRmuNjkUo8jdWA6+FZhTPwqYZtBmW4avn2Qg+nZYCSPxU1Xavt4QZPaAL0CqlEc7K/X0I8oJFkRLyxWqMOooCRYEusn7ONBOkmcW/MO4hZy2aHw5f2tJwYzeR2d2UKz9uZ+ShGiTc39ajYFmYucE/3q5c4szEnHhYPBhs8R2wdxXmKiWGlBpjbNpw5kcrf/CFFTYz81u+jEpYKeL6g1raqqpQoxNII1Vdd28yU1XyMVdQlp0gVp6uAadCSzTmQ6LubFQrq0k1libZ0lxPLiRBIrGKJRzSxUqgUTwlOlljerBVKulEmlWtvV1U2B6HnMPMlXkyUhjD5RGn9ac294nGCmNWDqYOcRMI92sRf1irDC2OQbIcTzeEjsEYjUAYLT9AsNIILQO7kjsqkBPgKHKJpXXqIk+0s2+VuIb8NbZnM0X7HfnVX4WRkPyniizf6CwXq9ET18Mvcoo/xVj+cNzwulNmrpzcOqjLWpHjrRr+WBrggBiKvbO+IvH+kw5Aab8qRN1D/12EwQUvgcVxjMzSGTj/ohSRTCHOVv1VK1opHabn7RKX4GO5dWtcnYJcCLBd8jULBo0oCQZBZeriDXRLnPAag4R0eTOxCLlSNsFOREWgdFCjZyDtVy3oTbLisqqpoLxAPHkLn3sUWCHNhKrhf4eo6xXMxgvMnjA9svFKheWoF/NWQZcCBahBEfO79y0EqMRaLAiIAX2PzZws8CadMgpJJ8vlj5qYLQApXrrHgB5NUCUbVYerZIBODi93eMQK+lvb8chWRASVB+9Dxy6uDMZeQXk7ngaa20xeTry5mynkkwH/okSJJ2+Gx36qD0a0l6gat18l4AeX2p94nPiPPflt2LKD8U7jq9h7O/UtuuUs5Xy2apquklXCpUq7hUKRarul6plXS8W8B/uW2HSs/XuMvXSJmomlbTsF4ySHW3oqmqUa6ZRMXFglr9FRp3II/fv3UX2v/m7tMizH/bd3+jCP//tfBeNnj/2Hmd0MjbUMXJE5HppCduUSmzFDHruPAlENqRTbGovKtWdaNa0xfLu19KXRtbGQsgr1YEFtXqq7ejHm9N7pD/jkn3IlvrjGER5vXd2BPJaC+htPwNWltgNN5kmnxHOc1P0tiSE+jPN7eF6t6c1IjGcPhnamwtWMk6M14AeZHD/ddy/L9HYE9we5/oNAA0v1k/acbXWqlPAV5f5sOA9FYfS1oS+aU/JC9QWL2YwKdMrZP3dP6ZY8LaR+Dm/1EGWvrnjqkn1wePr3NnkAYdHfwf8V6d0Q==</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_9572198988a54a0b9731fab9e0c97e51\")) .filter((elt) => !elt.dataset['step0']) )[0]; root.dataset['step0'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>\n",
    "\n",
    "\n",
    "\n",
    "<div style=\"display:none\"> <script type=\"application/octet-stream\" >eNrtfet62ziy4H8/BduTbkqxJIuyLpZle75c+pIznXQmmZ45vT76HIqEbCYUqSEpR26P/u+8x+4D7Cvso8yTbBUuJACClOykz+z3dTI9iQhUFQpVBaBQuJ36wY2VZrchOdv3g3QZurcnVhRHZN8K/LP9eZxc+mROkoT4l0P3aDQYz3vDUW/eH46G4+581COD4YAMR+Njz9k/P02XbgR/I73zjpsk7u1N8OulF0eZG0Qkse6sj9dBRtoA5xEsKFm44cTaWCbgThDNY0CZQ0p77i6CEHhbxFFMsSeWF4dxcmL9waV/JtbCTa6CqD2LsyxenFjdTm9AFhO1xGVC6osLouUqu8hulyCQxI2uyP4UWLghSRZ4bth2w+AqAi4C3w+B0jwIMwI8XAG1FPJJw2laMRQVZLeNbmfQvHdhJ9fxDRVUmfT96EWrxYwkQDCKs8bJPPZWaRPIzuLEJ0k7cf1glZ5YR8v1p5FkvynTSF7oZET/THhxJ5azXFtpHAZ+kVVTaicFSJKkur3UaY+ykAVLwFEMeWIt4zTIghjU5s6Ah1UGaTPX+3CVxKvIb3OWaUEmhmchwAIV1/eD6IrZlXeNZIMINNQmNyTKUlHYx8DPrk9Ae1kbmYOsiYWczcP444l1E6TBDA2nXK1f20HkkzWU3O1262s5i9c71jJet9Nr18eiu/R/WC1aoRZP6EECr7q5Qjlf4xq2vDDwPvhu5t5HY2HsokQvFyRN3SsiWY9o0ZvTQ9aXnGYJIakXL0k7WUXta5JAWuolwTKzqG3a7nIJPLgogcPYy0jWTgHHXdjne/gHik0zS3BhnVmNRtM6O7fu9iwL/puvIg9RLZ+kJAmglf9KfgZpHDew0QCAZSUkWyWRRVOfIJ3OPIkXDTeLZwDUshoLSnDR8WKfvEZRPska3WZzAtibvepivgMxZEe9oiDG6uw2Iynw+aDyBJE50kYqEflo8YIorQYl35mt5tC3cxReQYazjesX0X8Lz7SYe3HMWA5JZj1DY1q4yzffP30OljnRa3NFsmdgjEG0ilcpBW7cuOGKtJgZAiaiiRoixZmbkkvaGlpWPJ+nJGN8BHOLoVqnZ1ZXYFgSPFSnO+GpDLNI2VgkTIlE5PzMciqIyJx1QhJdZddW2+qVSDsdlbggxkTsZWlOkRX52GqYSTvNiYmPl2523QG5g8xyYs0SF0U5X1sO50fSdKJV6KIoYnrRnSJTDrDAyDWtA07eqkKyDiyHI8raYYVd1RXmPLQwx1zYrK6w3kML6+mFcfu/SFrWVcuaTc2N9jYC18l7knhpEF2/IUC9wcv7QG5pn/9XbvZhsHzpgm0n7seXQcT+xW9O4nt3Kcwyp55mLoxnb9E/8XkRDcDI3FVhwWjZXwXpd0EEA0ODZv3jH8yEYKhqrJvWISJYp5ZD2v0CL6/gWliWZs05AKWVgmeGxB5TYo9zGPxDAcL4qlEu9YBj/z3JQCn8axl/bKwZQMvqNZu5bW8kK87bviJH60zpAzCfyVPL0MSPFsPkX1lZHZ6RNXGmQ+a1VjJEL0q1LKAW7roh9M4Zak4qanqaQ+Rc/pv+Yc0OtIVtxjrb0zRP1stGbgKqDKAZFuYNtpODiarlqs/LAFGJkg5ZQ1bFcqgJjg0eIFjoawHX1GTypoetYyKZjsDSTCcnBi3G6aomkHev2NSXBHtZtHYYd+sa6yGnKWjxliWIGEeTT9ZWYfRCksWnyVR1QihXn0TxIojAx0hyGw6ihmQCpmprXR8XAWVB6u1aW6iIhoLoitoUpjTdqQwbFciFv5PaJHLFEFsaBbL4LTrSb7MEPG421qu+Wz6IiKFLGWLeJVezxqO7ZGM9urvCv2ab5jvjcIP+feKm4FZdPaxECQKnIxHMam4BotNzekNonwl00Z2RM+jB7yv83R318Pes6KQKtHPL6R0XsueVsekEyDaatAChkxhbEahWUZifh6/dDCZMEfhAoA/477YF00SalA+UoOQGmmlAHT7451SAcCcL0g4OmpqHlsQfAZ4DXgRTYSA5ufeM3HsgB7A5qfcyKT74xB8v3k/lVCgkW3eQ/zfEyxroXbwH3uGfoGU5LcnnKyxyUzItxqgfXAUZdZ5fJ8HCTVBVFxTW/sOc/rFb8NOZj0azPv05n4/mXUJ/9jy32/PoT3/YG/WO6c9xfzia+fTnsTcY9md2ixMkR6OR16M5M2/m99hPZzQj3twGGComna+3BFJ8lbPRHP9HsV3ijcgx52w2G3Eejv35sctTx8fjIf3pDWZdf8B+9sfeuJ9zNh/Nhj5jx5/5s2PG/pj4LhnknO3l3HkkDN/CLApYGk1YhjZpgZnJPLgqzVl86HF+isgzwBc9HLU9UDKdtrQsPoUJUujZAr+YyzCCrbxDFwbBnTMKLVkhNxDahIFLO/4QeteNweBrDBo07cmewZCgKGiII8oN+4H/NSf1NEddjWapYXG66ATntNnHRW6nF92WVfw3bSkZDk11yhmfBWPaLM3jVKF3MEIBwwBU2csnnLbe5D3u9iveS4mUgFJGmyB95b5ik8em3NBLEpc7vvuqz0AOB4XeYEDlAf82JcoPUiKI2JFE7Ezl8bdSLblinJIq67Gqypo2NRe8mJqDDl+AHmAic6sImmkQVQVyMQQWYD7eskq6K4vUNEx/LnXVjsyfrjkmYuf+6nlYlrGses21H6S67u9CdYVETUqoaj717bG+qTpbGp2uIj5w+Tjx5WMY7/Imqr8j4M4xCqb5PQ9tp/dV930V/mCVP1jptQ1sW6YjtT/nXpjVmVPZ6yxab67OU6t7b3V2f3/q7NbJvftgdd6brEGdsuoKt0UouKnqNp/j39/3MBZYIqHk4R/VBAzWJDiVTEpR3ebB9qLNsrY5kpa9dGGykPHosF0Kv32SR6m2KpF9Qfvaqex2chDgZwVswNBK/C/e5+f3PvWBsCJuYpJRHrZp0bhNSwrc3F8FOxsnnXvP4kq7xEnsazfJ0qe3zxE0n5hTucgTLpTfaFpOAbX1WtaROQcEO9gKMeQQffz3CAxoK+SAQ1KM/kMwjvDfwadhwr9DtJdi9inHmIPoBufxINC5C1qSGyvzgr+2HOsrLR5ZeE85dpasyBYrjMiVmwU3JF9CPC1WOAXMwr0Cd3vlK0sQus8mxWtw1TfH6WB/TLviZiddhkHWsG3N1WNIYrHytGRYPMfkNCDo5RJhoUwd70IhPJX7eJTyMqE7JC4TsiRull7Gc1ytXoWhMo4bAn8K2Yl1cBDoYx5v4WkG3LSsNPAJ54FzyViWAoIlGQLgK7pNh0sHYJsqcC452nWXK6PzVO5WDME1JjKNLWs3UVWM13Xl8mDjfUqloFqxRv+A9oUl76AYQbhyilGkpC35d6WHoDlDoj01672W8ihacpWqBj+pX7/Qqndh9v2KSZ4xw+AQTiuqa1CwXH9jxyO7RXKwiwF/8TQ+p6dh3Cuysxh2F2/FUgq6EWwp7knkv4j8wCNpQ49kBywdf6RgJxRV24AkOvQL3iXQTlisJQGSBT2BCVtYFuRc2OjH2FPqycCoQxI3tGVjo2V0lqv0WiBQRm1j7KlMUvfYBev5phlezQuO6q6D1J5O1YFPAJ9ZHCr9ECwvaT9ka0s9ErvvHt0ZwDcnajKJfEh8Z56N84JP71suwzNR3YZjtU1l4c4kXK+q53sXfTDl6YtmpQWRMnsU8R41shsLN/1AfCteZU37QWxehnH8YbUscSvWb6xvvrG+4rjBVRQnOEGkvWWNdqr5KleHmWq6mqUZ+Gi07eYmyHi7pGvV9lSbLgpOVdSqmWOJw1X0IYo/Rgp7FV6DhCcXVjUu7SJ7bIIm0aN3Z262HUQxt9nz7U0np/mQFlC2q3upjRWpsb6r1rbqrL6FVOhrUzURkYs7/eO5rU0q4pB0SJLEScP+mfEi9/02H0eMO7v4LgBWwPs4iMTcQ9lg+gSU/HZJvNIq7SU4f+7tz1EWhH9lG74bPsHYH92e3LJcCiZkJ41fbHt4kN3+NEtJckP37vBtsCRJCUUTWY0GuLhJQNJ8H7NQF0+/6E47gYT4BksH++uWBx++E/ulm3zAXfdnlsRu5+8rkty+BXfWy+LkSRg2bH3ntix5VreGvCohZkIkxPaiFaZaEIB0ErKIb0ijabLkkoA6fpBCHSL0PHRVtqy7Tb5TGGqRZk8imDYgf98l7oJIW8DNtGP2Q1ae8GPM27pnqyD0n/BN5t8FV6tE07xHQyWiztvsROXvclfqKouyXd6TP+E5zWMWwcYoD37JW13plvqnbkqG/QJISizBPmdxIgWUpsmQdNB6CeOSTlnLkHHw+AFGDHy6B53DS4ky7Fq0XQm0SJMhbw2Qt0bINIQBwDeAaxkyjhpCK1A8bRuI5OIGa+gGX5ME94AUCEqyJskV+Y462Lj48ELyfhWpVgFN9opOChsx1Vex10pWKQyVc3bGQBomKQDrUfQTD5KVlLdQaISDaCtZdiShkqjAzK5xYoK96rdscFhF6Wq5jJMMfCCfDvvN8l51aneX6CmphbJDIppVNmWh5ZKLQ5zh8n3wmOCtkgT6ajUxJUs6g+nKUxgtjpRv1C1MtiMiD7d6UlPbYIYFsEGTl18E/nN+kNUD8Z1v66f8K4kbvZ5qjbM4c8NncZhq9Y7Dv+EpKVpPZ1pksOqAVEtzOE0ApXobN9GlAAA4BbAUN+NBTNyOCzkdcJ3AoaI/qX8lwAqOmMgKv0SuW/H7MZA8oCKGsnCbf6ScochrzqjleIrB5ZJ7E3/UJAeW+wMJrq6zkuhudxXd7X1Ed/sJorutFx2vXPF7i+iKqkuyQ8RmhSnyIcfDoNpb7IF/oq4tFnq3qZaP1lnvICQNQ5JUufALKqcInI+pdABoT+fac2Halsrrdg0/9lYLaHgdLyFuRr4NCX41bAZq52eo6GeHnkLEjd+FaR5YPTwbIbYeKuDXVLI5PNWHGV4S6jOKi4ukZJ0pvHKqfAkVcht2z6csGjoJvt2aHkz5E9+VrQSJi4A9OJ84vKEKVddIWgIRI+ebYve7HG+uW7cybs4zonCeVd+Vb/KHAkvwC+ksgLSzv7Q1nG/t17eFlxaX2YYYevzjySw1lZhnqrsSckR3XYNIM0sMmxV1xuRbXsgoqYHvZqf7jVEguN8ed27rwjEv3nMHku5+fylI1pxtKv6Y+G5hqa1ChK1CKC2r23EGzcmu1VF4wsQDPL13WBzK2xoJ54tleELIYDpBZFDhfeyMnTxayKd0tlWq2xnsoJEaDbexOlTDyCn72hKB2TMc7QDn/YV+fFKK6dIOGGaUWg+sHLbISRycyTMEuSN+XN9RT3QO+RSiGE8kdwQtkPVNXW1DxW0Bf1sPz7uxYui38Ex1SKB3TvDE0p20eMK9Agabp1aCKz+lCQLgU8f5IpfXtGx0Isp4pnjDvAOw/kgXb6wT66uviuwKeoZt7HL8QBtb7rG9fa+0aKfYqGKD5p/MmfJv3Mgjz+JVlMm291CnintEwrbAvzmQlJs7OJha+DkUTDhEKmylVSsOm2q+RY8g83F+JjlquFZcWoTVPlXZlKuhVlOTncp7eyvv0DQV7nRaKDOg0qgQWrMev2LVWfucgb/1oXrdeKN1t5rtnOXiMG5G0EorlSWPG9ZeecuG0g+sq832IZMoOv+pMVnsog7OijlNpb1WWuu63lpRnGvVVtd1tqp8rLfY6brOSittdG200XW1jaGQ0ELNUmrWIRvNc2dzUdzXtW6U6zqj3KssoWqwVv8xdOGdMIjI3/ikxJnUAKZZEn8gFSv5VZSfuUsETv++chOyFfo/Yupq2Qtc3bV/0yF2r3Zg45U1bThhq+j5ObE2umQtQ++3DSbPfSzboDO1/vhHdFNxY0INhtSvVqFUDammcdSpHEedL+Pol3G0Zhw9/3zj6N5ug6dTMXg6XwbP3/Xgef4ZBk/6txwMK65TIZkSp2hE5KP4rW5WkjJwQDcFOZp8HVuwUBVpKyjdM0qG+022R8bkyF1pjVaPCOaSYPsSXuOyFl52cFt1xoHOk6XriGjwgQZgxECGV3w4E8PxehnrtgJLFQhucsa7axCf/8TUc7khFzH+4hKlKUDuqd3tuTJESdFtCctwFY4cy6z3wsSpuXQVZlLM+7NFVsTiMCNSxP+hge4UTrE4b2pInNKb3DN8oyBt1CgKWjvm88umShcpBGr8ppA+9vPnbLLUbpcrXre4lMNQZq3t3TwP6sX+KlylKjy9+inHoV8ynl6/4vNrQW9SOusaJ752h5eMeMj5Lu3RLy4hQwIHhjUf0VAwn7cPhnReLBap3aXBrOX+97c3EWh5sgjh02gitw83kdudTGSrv6rbiIxQayTlGj7MSBTE34+RiEvW9PhoyzKGOVucnSJoOa1doCzuw8R4vnRJULEBC0OXkf8MJpx+5TKgH9zYTfUixSCiVKV1OX1FhRV8f/qUsoLGVvzEAl9Bmt7ema9G2tKtqPbECJqvRO4Ai/cCf0evBaazeXExcAV0lrhRijvNf0qCKxYAyOIl9AHzKvowcL1O4iVJstuGHSzcK9JOCFp/EF3h/S50zw0IybebOxBot8Xto+1f43iBBJwdEfGqsTa9BTkFz4Ri9pdrW4ibqUMV9TvPDb3GjZs0tHLRa350Jy8Tb5ZrcSRQppRrYjdSDLyClriblp4iQJmBe2ALY2H4OytJBlfkRC9JRuGYWLXLi8+60PkFvz9CWXmtH92VOv4Nvb2xMyAL7GehvqLCFfT+Ei8lcuudyJV2MYThj+6MhPLmDr6qRNNfa+ce+NiC6+8LaBcUhobU6NJ5iJ/K+jlNERaEN1+9xYaE0l9K8TQZil2T/JRetY1w3Y4DNZAuTDZioQ2ze4+krsimosE72xy6LNvodobYuZaV2ESLYxlqk2iKsyib337PEK2PdqS6vtc0SUJuEeK+ZqPIQtkcqxshn7VI7U/XF9cUI+R0u1+DvT26C9D+qPmZ8Xh/ItV2Gye661pEKA3MYRT0GevkxSSCZphYQdgneP06ykzqESyjoXMrz2e0pSFLzc9bGNtdJGdufvO9VP/NJqX0mkmcAXnsNtvjrk+ubCNtQ7/MLSrBbt9YTKKMHztbnMlS29bDLf/eFlx2rGtN2GCcqkx2sM491YMGU/3xwSZRoG9tXRLoDiZUgp4VvX81kLCCOhjo+8XakI36roZUTDc3ynZMrRKNWLFKzZTl1l9QfmgXYaAgDcYM5o4CnUj1aOExK0yitDfNqh2ANPDyNF6TtNqD/6Q5QlFAxwvdNP0xSLMOeCzg6UbzGEXJ32DIPaf7hIc4GLvisHabKJ8FsV3FbzlTmuFrKwkS67tUG98ksaUrnLWyOI3Gu/+KHt0VbWRz8U7bwIOvX5RYqyqUPpshtUqGzKOols1e0rC1XLEVpywlDsD2iFVnu2tV3I4GkGZkKa9tmEXBxMlQtknNnlq2JidmNw+UE0PO5UQfRbG1zGoxcYAqMYnsSjFxAF1MeXJ5hoX+ar/rXePVKikfIIoRok6+jGhZdh4GTiHnGfQ3+EoJfauC3GSKw16SBQB0oKpXJGNJRehDs61qwD11pbp62zK/X0KnpDRvKS5UEf/XVQ59z7d4awN2RAS6S2EbLZNQ9Hb1EORyp4tRnBAofMYul1Ms+h+R0qEb6ezc/1iw43B26eA63vMNXS2e4scbbToDeqDewRto5BUCtQY/LfkwrvCfl70L6zElIXecKvHcovJLw+XzMuhOlTuCHbuBAtnYEUjZrKXbjiGHNnK7Z8jizdt2o9typqiV4UCVTIJkT7IsCWYwmDdsquGWrFotEDeP8Zmtzxfg4xSNI7cGUj+pF1CiN/8fMb6c1daJ8OxCAOZ8+4DO5+mx5HncLMcx8jeaKhtY2QnbSSCCMJPIK+iosJ4i1Z7oPGgD+efjATujMg94GZYGdL9wXXn6oRHaOVpXpsQ9sPwFp88YhxY0q1zMHEANz0l3rK9f0kfdYODQejoMAOHGKRrrR33BOPQU3/MKoqtnYQDMvFHOBEtLPhFMi94IcT26E5T4VKWdk6axFxDUO+OKkIgBSD5/xU6Q3MUVYToJR13i4SAdOkfQJlA51wZ4OdZqniKhtOVj2lgL6udyGWqktgiTLeHk+ML6zgvRsRTt5q1t7Gp+vso9C1Q0aJCiadEsDL432t3OAP0vp9MjC+2GiE+poaXaCY/floylEAK3Fjw45V3Ll+wZ9qI8mHRXDZzou0oUO5+vQlF5QbvOkov57TY75kRzBNzOs1WeEjfqqal5dlI82yGgaJVbojD6pbzGkcVLExYkF0jwoeBQuZ4Up2dzLJpR4NFPBVM8KVlGZTkFLvuWkDeGx1nqlh0kO6Ch37aliETujuoWHErhLvYaJo3A5QXg0opEHz6V+FYFfanrVCUod50JXw3aRixfQZCp8SCORI6lFPQ22ql0HJiAGrovdMtQWuxrYVa7WsIwQzD/uyRevOWeqfkYobI7AK868V+v2eUZfD2cpjY+BpEff+z45AZmGLRUCqTczF8Bg09dKZt79GIc61AuCj+3Fac+/rPMAZ7471dptmAxQK2gSqrVz+14afp6/Rc67aMvlCQpO8/f0H1oRQ4yllZ3laCjVwexFfTHWiWa1tfSexpnpetNPm1J13z/5b1ouqsstivURNcs0Q7lGsqb0B4bNVnVqLYsOEMbowVKt1bJHpayhMb4++hm3rVkt3qL4ZtSiB+otwxzs1og+kvMbbxrJIQGlOkTpY/uKgxv4y+xC9LCIiGf1pfKt1jZ/M4YLQTgXdNJYytHl0YjU38gj1ZaxaW4hfjBCi4HHqpK3ZQmkdVBC7WSNZ1bfntc6LN1Zok8u8oJkgVofZ33FM2lXhKH4VPFJ7ujg66pBHzLkHLQsmbk2r0J8BVXG+9XcqPM3uhlyBNpWs6LKIv/GpCPjTsDOtAMY+8DpETEBSMqCG7UQ+9leS7iVUotA2Wqx9DE1Ue4zxQkJ284xdAWq9x/tqzi4xelTxOYhhPQ+XSUNUv+Ui8N/MYRka6P1GZxVYBVN0HnpyEvb7Q7EvPNR3nNdimUCrqaPbYWrXjpW6abdNMsPRBdsYZWKiPTvJftRdADH9uLQHWoC2LmayeVsqsFXHNlTnOygyHoopaCE+AaAcRQz0OPifhioSynzEyTJR/kBMzIP4gogYbN003o9EKxojlAS5f5OLe61jffKCKTgQ80YDb1kjhWJ4iatPjWB1u57FaD4bNjbea429KxWi1Rdc12DJM3I5tGDnYq+qCq6KrKKhLZVGjql1xTP4jJeY2qfslVlUPLuvrBMJvXeKPttl5XYr34E5X1y6coq9y93ENXv9xDV8XiuF15BGSn4StmHoE+eu04xOw0wGxlhMbpto2hb8RRgweOpAK/PJyWT8UgwMMHx8tiVGSFTvTrVC9vyte/1nNhvtkjD4CqI4/9DDOIf4LBcs0at1yGrFemdN2J6c9ug5WlPpdlPlKknqjaJpOKg1glNP1RAPMNITvZ6sdrQkKTrYr+0Q0zKLcc2PJJmLm/4BSD4TY7LKXgCrH53fLPydwFw9FDosVD1CWnvYkPLxT5jLb26E2tpy/LpAyI+9o1dvnUAEYNhPsR+A53nLLn99PJiAqhiX4QuszRudXfVrv2mdXX7U8p9bSCXbzNpnSdjcqu8nmgnmA0HfWrrMep1d5akYNtFTmvqkgQ3asi7e0VMSxmyiR2nwOXfzxs6aU8qbzTI7m0YXoU9xc57ochXOtQs+HHWp/X0CSkQCtxXBZslgr7T32553OWZprLFvoy3Fa7dNM0uCEn7PWWjbImZtpFUadAYwCjdGEtC1T9FJHn/M2ez3ddrXq7kHzRqGHzKwtecCCRYoDLw8UyKEucqNczla/Q2/0SvdI1euoleOVr80r5jIVM3y1WfTFe+V4k+poEe7SBD/541wRYCW6JOJFeWQLL4aOudjI2P1j0gF2klUd4HM9d2lVQxemdWrAFjTgyndvdzmhAFpWwkscaRHh7R1udOZfWFcxHTczA6BzPQ3oY2GbXP1fD1p82Mkis8hyFDom1ymfmdq8zsHeK724LX1fVeZVhgVRJy7V2ZGOv7mxYfshLaQS19iGDVBmHDFOxAXlPW3T+Cw4LpUvMMefbsDQX39Hiq2otyMrOEU/S696T9y6XwIyHXywMymMTgKlmvpzp0H1eMBKJyoqbBjZk8a6Sfu3m7RK0CK60B3U8F7NYx6mDU7a/eySSL9YpF63Yu2MCVKdL+oCw0e2BjQC/H5Ng9f23WQWP9fzbTUIM/KpVVG1ZynVZvsOfn/G9428FaD7RSTmJ+XGG6/RPTIkIvZnsbZrUC8uuA+oCvInj7FXsk0azcx2nGcw251HaEQEocSMj/JycHoLnHCyz89PDLCEk9WAIaCerqH1NEnJ+ihvcLbrD6mx/Hoc+PuVxGQHl/fNTKqfzU7qwZKHncLbvXRPvA9Rh34hzmcVXVyGiHlIklTy97ePSnc1gFrxfl/VNmE3eu+sOFYTFr4Vv9FtN61///J/dTtf6v/8H/7741z//N32k91///F/w79T6lSTxSR8qDJTP8394LeTiYJofXZI1pPjE11jBc9xgKP6lh4oH5dXmQybuzuR5QuiXubloyNrbF/vnb8QIzBTW6XQE30ZlUU1yVYB5hoFHTfUw9jKStVPAcRf75/l7Vszcqc2wLzSZSR6ymNNr/GtNasLAOsDD2ziOxKsXnES25EEB7A2Vpz4adkYWS3QnkA5JEhAYOG54IKRwMdmrBf/x9qdXHTrBbSDBDj+3rdNjdbebSlsGaoiSv/phqe2gY3qzgrFbPHiBbXlLIxF1EWpUqrRfo5f3aQz2c7ePexb3T6z95ywsll98A149BvNgHmu5dIG+2bF+QMfykKXjViv2WgENBO63rH3pjQKk+MT45+wsB6XPICAkb0eYoT05QOn8+dunT/7M8KQ7DiDrbt/twj8OTBX2i6OtkHJxt4+7+hEbICCXtjT6CVBW98TqYyrd0g+p0FD3wdrhV38zhd+3Ci1M0U4EiWT1liDKkahTcZ8QlgRjHZaDXcL+wl3nv3kwEr7ZLGg/v7o5B8lvcM5TxNvFyMXFeMh2zR/RR1Wdoz777Dv0cziCnCH+n34ew88RPg/LPsfHANzFd+zY27Nd/D4aw18Oe2rWwYQRPpY4YAk9SOh1jyBhzB517eP+fSyhd8Qguj3cwU//YoUMEX2Mb+t1uzTBwX3+oy5NxYTRAFnGko4pxhH+7B3l30hs3M/BscQhVnpIn409Amp9pOg40ykagrLDHUQ0YlZVFShm1iK0xl/G2xc41GTQVvcxkFELdqGC0Es3qKmD0RSGWLx1hdYmEoTxbStjup2N/4os6LlUOJa5mW7kzoR3HPATholzfWhi/+w4RmqD1j+s53T7y4n17PXPVpcT++Yqm5hLYQzgYEW9JujE2OT4xMLFnH0r8HEwTy6Lvq3f87o9ZzwbOU5/SPwZcWeONzvu9p3umAwdzt7OA3njotvB+9HF/8Gu6IspZ7xjapp4vy/TwNmAOLPZeOZ6fZ+Mjoczx/EH4zlx3KOeMzIy/f+734OxcPB82tAxHVPfZ0h9H/jGFgvej9NxpigR6gA5wy8e0BcP6PfjAX3/p/Grm5FzmyR9pxc9O/zl1bcfXv8pIdd/Gf588OL4cPCn/o+ve7dvb0aDZNF/ebh88nF1018uv/9x9P7PvQ/jRXYT3ITH7vejg+jPr/oHT5Y/vr65eqgHpf+/wqNy0KNqWcy16te6Vk7ZtXLu5Vp9mpv2eZyyNnRRvf5odDTo9o764+PB0SB30oxZ23w2p9sdjIfH/YHTH4/H6E4pPpyR5heX7vfq0rWsbYU4X/zGz+Q3Dgfd0WDeH828PriQo5HbHx4djTxvOO577nHP/W1dMIuCEf+BrpgXLzCYe1nnH9E8AA/dZYqZ6LaZdKLR+IPFgT6Li1fLULUyf2NXkDv6F8D3cfeoP+iNoReBj6N+d+SMnSP6Me53j3vjodWypI68eFr+AlJ7Q5hy9PvQgRU9OYMf9AfDo9ERJzvqO8O+ZcnITqfbO3aOuwgPIMPR+BimLOwDesvjftfhJQ/AIscD6VF7ZLvfP+oeO8NjhtwdHOPsh8L3oRLHI4tR6ncHMA06PppCV8oCx8pUpkL6hob1yU4uu0XhhU+vcCv8sqF7NIJpT2846s37w9Fw3J2PemQwHBCUiEe7u92nejLw1imWDLy1M5hOuC8eewm430ZvXDjMKCsOxME1F3m/k4vzErP3S6jPwpguX4rvjocJtBj6GsOkyEkIdHMe+VuQXTcU9JzoPHGv+NkobU3oOf/8jkNgHQS0sq4gCGO+4VhlrlztWCV751Gce8opwydfjXp6+wKIC2y88HzCXgiNV4lHntMLBSpk+AfszfetA0tD59d1yGLJqXXmQZKKsmnNAKHILWYnG1UTRiFvm4z8P+G5qgk=</script> <treescope-run-here><script type=\"application/octet-stream\"> const root = ( Array.from(document.getElementsByClassName( \"treescope_out_9572198988a54a0b9731fab9e0c97e51\")) .filter((elt) => !elt.dataset['step1']) )[0]; root.dataset['step1'] = 1; root.defns.insertContent( this.parentNode.querySelector('script[type=\"application/octet-stream\"]'), true ); this.parentNode.remove(); </script></treescope-run-here> </div>\n",
    "\n",
    "\n",
    "## Multi-process checkpointing\n",
    "\n",
    "In a multi-host/multi-process environment, you would want to restore your checkpoint as sharded across multiple devices. Check out the [Load sharded model from a checkpoint](https://flax.readthedocs.io/en/latest/guides/flax_gspmd.html#load-sharded-model-from-a-checkpoint) section in the Flax [Scale up on multiple devices](https://flax.readthedocs.io/en/latest/guides/flax_gspmd.html) guide to learn how to derive a sharding pytree and use it to load your checkpoint.\n",
    "\n",
    "> **Note:** JAX provides several ways to scale up your code on multiple hosts at the same time. This usually happens when the number of devices (CPU/GPU/TPU) is so large that different devices are managed by different hosts (CPU). Check out JAX’s [Introduction to parallel programming](https://jax.readthedocs.io/en/latest/sharded-computation.html), [Using JAX in multi-host and multi-process environments](https://jax.readthedocs.io/en/latest/multi_process.html), [Distributed arrays and automatic parallelization](https://jax.readthedocs.io/en/latest/notebooks/Distributed_arrays_and_automatic_parallelization.html), and [Manual parallelism with `shard_map`](https://jax.readthedocs.io/en/latest/notebooks/shard_map.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Other checkpointing features\n",
    "\n",
    "This guide only uses the simplest [`orbax.checkpoint.StandardCheckpointer`](https://orbax.readthedocs.io/en/latest/api_reference/checkpoint.checkpointers.html#standardcheckpointer) API to show how to save and load on the Flax modeling side. Feel free to use other tools or libraries as you see fit.\n",
    "\n",
    "In addition, check out [the Orbax website](https://orbax.readthedocs.io/en/latest/index.html) for other commonly used features, such as:\n",
    "\n",
    "* [`CheckpointManager`](https://orbax.readthedocs.io/en/latest/guides/checkpoint/api_refactor.html) to track checkpoints from different steps.\n",
    "\n",
    "* [Asynchronous checkpointing](https://orbax.readthedocs.io/en/latest/guides/checkpoint/async_checkpointing.html).\n",
    "\n",
    "* [Orbax transformations](https://orbax.readthedocs.io/en/latest/guides/checkpoint/transformations.html): A way to modify pytree structure during loading time, instead of after loading time, which is demonstrated in this guide."
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "formats": "ipynb,md:myst"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
